CoChat
Sign Up
Cited by 73Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareMultimodal Machine Learning Applications

A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions

Open access
Liu, Lei, Yang, Xiaoyan, Lei, Junchi, Shen, Yue, Wang, Jian, Wei, Peng, Chu, Zhixuan, Qin, Zhan +1 more·2026·IEEE Transactions on Knowledge and Data Engineering
DOI 10.1109/tkde.2026.3709941
Open this paper in CoChat — free account

Abstract

With the advent of Large Language Models (LLMs), medical artificial intelligence (AI) has experienced substantial technological progress and paradigm shifts, highlighting the potential of LLMs to streamline healthcare delivery and improve patient outcomes. Considering this rapid technical progress, in this survey, we trace the recent advances of Medical Large Language Models (Med-LLMs), including the background, key findings, and mainstream techniques, especially for the evolution from general-purpose models to medical-specialized applications. Firstly, we delve into the foundational technology of Med-LLMs, indicating how general models can be progressively adapted and refined for the complicated medical tasks. Secondly, the wide-ranging applications of Med-LLMs are investigated across various healthcare domains, as well as an up-to-date review of existing Med-LLMs. The transformative impact of these models on daily medical practice is evident through their ability to assist clinicians, educators, and patients. Recognizing the importance of responsible innovation, we discuss the challenges associated with ensuring fairness, accountability, privacy, and robustness. Ethical considerations, rigorous evaluation methodologies, and the establishment of regulatory frameworks are crucial for building trustworthiness in the real-world system. We emphasize the need for ongoing scrutiny and development to maintain high standards of safety and reliability. Finally, we anticipate possible future trajectories for Med-LLMs, identifying key avenues for prudent expansion. By consolidating these insights, our review aims to provide professionals and researchers with a thorough understanding of the strengths and limitations of Med-LLMs, fostering a balanced and ethical approach to their integration into the healthcare ecosystem.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tkde.2026.3709941

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior research indicates that although agents can assume specific roles, their fundamental abilities may be insufficient in the domain-specific knowledge and skills required to function optimally as clinical specialists in complex clinical tasks.

    “This observation is consistent with prior research indicating that although agents can assume specific roles, their fundamental abilities may be insufficient in the domain-specific knowledge and skills require to function optimally as clinical specialists in complex clinical task26,27.”

    From Enhancing diagnostic capability with multi-agents conversational large language models (2025) · cites this paper as [26]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 237

  1. [1][1] S. Xue, Y. Wang, Z. Chu, X. Shi, C. Jiang, H. Hao, G. Jiang, X. Feng, J. Zhang, and J. Zhou, 'Prompt-augmented temporal point process for streaming event sequence,' Advances in Neural Information Processing Systems , vol. 36, pp. 18 885-18 905, 2023. -
  2. [2][2] Z. Chu, H. Guo, X. Zhou, Y. Wang, F. Yu, H. Chen, W. Xu, X. Lu, Q. Cui, L. Li et al. , 'Data-centric financial large language models,' arXiv preprint arXiv:2310.17784 , 2023. - · arXiv
  3. [3][3] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, 'Bert: Pre-training of deep bidirectional transformers for language understanding,' arXiv preprint arXiv:1810.04805 , 2018. - · arXiv
  4. [4][4] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, 'Exploring the limits of transfer learning with a unified text-to-text transformer,' Journal of Machine Learning Research , vol. 21, no. 140, pp. 1-67, 2020. -
  5. [6][6] Y. Tian, R. Gan, Y. Song, J. Zhang, and Y. Zhang, 'Chimed-gpt: A chinese medical large language model with full training regime and better alignment to human preferences,' arXiv preprint arXiv:2311.06025 , 2023. - · arXiv
  6. [8][8] J. Chen, X. Wang, A. Gao, F. Jiang, S. Chen, H. Zhang, D. Song, W. Xie, C. Kong, J. Li et al. , 'Huatuogpt-ii, one-stage training for medical adaption of llms,' arXiv preprint arXiv:2311.09774 , 2023. - · arXiv
  7. [10][10] H. Nori, Y. T. Lee, S. Zhang, D. Carignan, R. Edgar, N. Fusi, N. King, J. Larson, Y. Li, W. Liu et al. , 'Can generalist foundation models outcompete special-purpose tuning? case study in medicine,' arXiv preprint arXiv:2311.16452 , 2023. - · arXiv
  8. [11][11] B. Min, H. Ross, E. Sulem, A. P. B. Veyseh, T. H. Nguyen, O. Sainz, E. Agirre, I. Heintz, and D. Roth, 'Recent advances in natural language processing via large pre-trained language models: A survey,' ACM Computing Surveys , vol. 56, no. 2, pp. 1-40, 2023. -
  9. [12][12] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, 'Roberta: A robustly optimized bert pretraining approach,' arXiv preprint arXiv:1907.11692 , 2019. - · arXiv
  10. [13][13] K. He, R. Mao, Q. Lin, Y. Ruan, X. Lan, M. Feng, and E. Cambria, 'A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics,' arXiv preprint arXiv:2310.05694 , 2023. - · arXiv
  11. [14][14] J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler et al. , 'Emergent abilities of large language models,' arXiv preprint arXiv:2206.07682 , 2022. - · arXiv
  12. [15][15] J. Wei, X. Wang, D. Schuurmans, M. Bosma, brian ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, 'Chain of thought prompting elicits reasoning in large language models,' in Advances in Neural Information Processing Systems , A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022. -
  13. [16][16] J. Liu, P. Zhou, Y. Hua, D. Chong, Z. Tian, A. Liu, H. Wang, C. You, Z. Guo, L. Zhu et al. , 'Benchmarking large language models on cmexam-a comprehensive chinese medical exam dataset,' Advances in Neural Information Processing Systems , vol. 36, 2024. -
  14. [17][17] Y. Cai, L. Wang, Y. Wang, G. de Melo, Y. Zhang, Y. Wang, and L. He, 'Medbench: A large-scale chinese benchmark for evaluating medical large language models,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 16, 2024, pp. 17 709-17 717. -
  15. [18][18] R. Yang, H. Liu, Q. Zeng, Y. H. Ke, W. Li, L. Cheng, Q. Chen, J. Caverlee, Y. Matsuo, and I. Li, 'Kg-rank: Enhancing large language models for medical qa with knowledge graphs and ranking techniques,' arXiv preprint arXiv:2403.05881 , 2024. - · arXiv
  16. [19][19] J. Li, Y. Sun, R. J. Johnson, D. Sciaky, C.-H. Wei, R. Leaman, A. P. Davis, C. J. Mattingly, T. C. Wiegers, and Z. Lu, 'Biocreative v cdr task corpus: a resource for chemical disease relation extraction,' Database , vol. 2016, 2016. -
  17. [20][20] S. Karimi, A. Metke-Jimenez, M. Kemp, and C. Wang, 'Cadec: A corpus of adverse drug event annotations,' Journal of biomedical informatics , vol. 55, pp. 73-81, 2015. -
  18. [21][21] R. I. Do˘ gan, R. Leaman, and Z. Lu, 'Ncbi disease corpus: a resource for disease name recognition and concept normalization,' Journal of biomedical informatics , vol. 47, pp. 1-10, 2014. -
  19. [22][22] D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits, 'What disease does this patient have? a large-scale open domain question answering dataset from medical exams,' Applied Sciences , vol. 11, no. 14, p. 6421, 2021. -
  20. [23][23] Q. Jin, B. Dhingra, Z. Liu, W. Cohen, and X. Lu, 'Pubmedqa: A dataset for biomedical research question answering,' in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , 2019, pp. 2567-2577. -
  21. [24][24] A. Pal, L. K. Umapathi, and M. Sankarasubbu, 'Medmcqa: A largescale multi-subject multi-choice dataset for medical domain question answering,' in Proceedings of the Conference on Health, Inference, and Learning , vol. 174, 2022, pp. 248-260. -
  22. [25][25] A. Romanov and C. Shivade, 'Lessons from natural language inference in the clinical domain,' arXiv:1808.06752 [cs] , 2018. - · arXiv
  23. [26][26] A. E. Johnson, T. J. Pollard, S. J. Berkowitz, N. R. Greenbaum, M. P. Lungren, C.-y. Deng, R. G. Mark, and S. Horng, 'Mimic-cxr, a deidentified publicly available database of chest radiographs with freetext reports,' Scientific data , vol. 6, no. 1, p. 317, 2019. -
  24. [27][27] P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. EliassiRad, 'Collective classification in network data,' AI magazine , vol. 29, no. 3, pp. 93-93, 2008. -
  25. [28][28] S. Joseph, K. Kazanas, K. Reina, V. J. Ramanathan, W. Xu, B. C. Wallace, and J. J. Li, 'Multilingual simplification of medical texts,' arXiv preprint arXiv:2305.12532 , 2023. - · arXiv
  26. [29][29] Y. Liao, Y. Meng, H. Liu, Y. Wang, and Y. Wang, 'An automatic evaluation framework for multi-turn medical consultations capabilities of large language models,' arXiv preprint arXiv:2309.02077 , 2023. - · arXiv
  27. [30][30] X. Shi, J. Xu, J. Ding, J. Pang, S. Liu, S. Luo, X. Peng, L. Lu, H. Yang, M. Hu et al. , 'Llm-mini-cex: Automatic evaluation of large language model for diagnostic conversation,' arXiv preprint arXiv:2308.07635 , 2023. - · arXiv
  28. [31][31] J. Xu, L. Lu, S. Yang, B. Liang, X. Peng, J. Pang, J. Ding, X. Shi, L. Yang, H. Song et al. , 'Medgpteval: A dataset and benchmark to evaluate responses of large language models in medicine,' arXiv preprint arXiv:2305.07340 , 2023. - · arXiv
  29. [32][32] L. Liu, X. Yang, F. Li, C. Chi, Y. Shen, S. L. M. Zhang, X. Ma, X. Lyu, L. Ma, Z. Zhang et al. , 'Towards automatic evaluation for llms' clinical capabilities: Metric, data, and algorithm,' arXiv preprint arXiv:2403.16446 , 2024. - · arXiv
  30. [33][33] Q. Lu, D. Dou, and T. Nguyen, 'ClinicalT5: A generative language model for clinical text,' in Findings of the Association for Computational Linguistics , 2022, pp. 5436-5443. -
  31. [34][34] G. Wang, G. Yang, Z. Du, L. Fan, and X. Li, 'Clinicalgpt: large language models finetuned with diverse medical data and comprehensive evaluation,' arXiv preprint arXiv:2306.09968 , 2023. - · arXiv
  32. [35][35] R. Luo, L. Sun, Y. Xia, T. Qin, S. Zhang, H. Poon, and T.-Y. Liu, 'Biogpt: generative pre-trained transformer for biomedical text generation and mining,' Briefings in bioinformatics , vol. 23, no. 6, p. bbac409, 2022. -
  33. [36][36] Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon, 'Domain-specific language model pretraining for biomedical natural language processing,' ACM Transactions on Computing for Healthcare (HEALTH) , vol. 3, no. 1, pp. 1-23, 2021. -
  34. [37][37] X. Yang, A. Chen, N. PourNejatian, H. C. Shin, K. E. Smith, C. Parisien, C. Compas, C. Martin, A. B. Costa, M. G. Flores et al. , 'A large language model for electronic health records,' NPJ digital medicine , vol. 5, no. 1, p. 194, 2022. -
  35. [38][38] K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl et al. , 'Large language models encode clinical knowledge,' Nature , vol. 620, no. 7972, pp. 172-180, 2023. -
  36. [39][39] T. Han, L. C. Adams, J.-M. Papaioannou, P. Grundmann, T. Oberhauser, A. L¨ oser, D. Truhn, and K. K. Bressem, 'Medalpaca-an open-source collection of medical conversational ai models and training data,' arXiv preprint arXiv:2304.08247 , 2023. - · arXiv
  37. [40][40] C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao, 'LLaVA-med: Training a large language-andvision assistant for biomedicine in one day,' in Advances in Neural Information Processing Systems , vol. 36, 2023, pp. 28 541-28 564. -
  38. [41][41] J. Wu, X. Wu, and J. Yang, 'Guiding clinical reasoning with large language models via knowledge seeds,' arXiv preprint arXiv:2403.06609 , 2024. - · arXiv
  39. [42][42] J. Wang, Z. Yang, Z. Yao, and H. Yu, 'Jmlr: Joint medical llm and retrieval training for enhancing reasoning and professional question answering capability,' arXiv preprint arXiv:2402.17887 , 2024. - · arXiv
  40. [43][43] Y. Gao, R. Li, J. Caskey, D. Dligach, T. Miller, M. M. Churpek, and M. Afshar, 'Leveraging a medical knowledge graph into large language models for diagnosis prediction,' arXiv preprint arXiv:2308.14321 , 2023. - · arXiv
  41. [44][44] D. Varshney, A. Zafar, N. K. Behera, and A. Ekbal, 'Knowledge graph assisted end-to-end medical dialog generation,' Artificial Intelligence in Medicine , vol. 139, p. 102535, 2023. -
  42. [45][45] Q. Ye, J. Liu, D. Chong, P. Zhou, Y. Hua, and A. Liu, 'Qilinmed: Multi-stage knowledge injection advanced medical large language model,' arXiv preprint arXiv:2310.09089 , 2023. - · arXiv
  43. [46][46] Z. Bao, W. Chen, S. Xiao, K. Ren, J. Wu, C. Zhong, J. Peng, X. Huang, and Z. Wei, 'Disc-medllm: Bridging general large language models and real-world medical consultation,' arXiv preprint arXiv:2308.14346 , 2023. - · arXiv
  44. [47][47] L. Yue and T. Fu, 'Ct-agent: Clinical trial multi-agent with large language model-based reasoning,' arXiv preprint arXiv:2404.14777 , 2024. - · arXiv
  45. [48][48] Q. Wu, G. Bansal, J. Zhang, Y. Wu, S. Zhang, E. Zhu, B. Li, L. Jiang, X. Zhang, and C. Wang, 'Autogen: Enabling next-gen llm applications via multi-agent conversation framework,' arXiv preprint arXiv:2308.08155 , 2023. - · arXiv
  46. [49][49] S. Hong, L. Xiao, X. Zhang, and J. Chen, 'Argmed-agents: Explainable clinical decision reasoning with large language models via argumentation schemes,' arXiv preprint arXiv:2403.06294 , 2024. - · arXiv
  47. [50][50] A. Smit, P. Duckworth, N. Grinsztajn, K.-a. Tessera, T. D. Barrett, and A. Pretorius, 'Are we going mad? benchmarking multi-agent debate between language models for medical q&a,' arXiv preprint arXiv:2311.17371 , 2023. - · arXiv
  48. [51][51] A. Lozano, S. L. Fleming, C.-C. Chiang, and N. Shah, 'Clinfo. ai: An open-source retrieval-augmented large language model system for answering medical questions using scientific literature,' in PACIFIC SYMPOSIUM ON BIOCOMPUTING 2024 . World Scientific, 2023, pp. 8-23. -
  49. [52][52] C. Zakka, R. Shad, A. Chaurasia, A. R. Dalal, J. L. Kim, M. Moor, R. Fong, C. Phillips, K. Alexander, E. Ashley et al. , 'Almanac-retrieval-augmented language models for clinical medicine,' NEJM AI , vol. 1, no. 2, p. AIoa2300068, 2024. -
  50. [53][53] M. Li, H. Kilicoglu, H. Xu, and R. Zhang, 'Biomedrag: A retrieval augmented large language model for biomedicine,' arXiv preprint arXiv:2405.00465 , 2024. - · arXiv
  51. [54][54] M. Jeong, J. Sohn, M. Sung, and J. Kang, 'Improving medical reasoning through retrieval and self-reflection with retrieval-augmented large language models,' arXiv preprint arXiv:2401.15269 , 2024. - · arXiv
  52. [55][55] H. Yu, P. Guo, and A. Sano, 'Zero-shot ecg diagnosis with large language models and retrieval-augmented generation,' in Machine Learning for Health (ML4H) , 2023, pp. 650-663. -
  53. [56][56] C. Long, D. Subburam, K. Lowe, A. d. Santos, J. Zhang, S. Hwang, N. Saduka, Y. Horev, T. Su, D. Cote et al. , 'Chatent: Augmented large language model for expert knowledge retrieval in otolaryngology-head and neck surgery,' medRxiv , pp. 2023-08, 2023. -
  54. [57][57] G. Xiong, Q. Jin, Z. Lu, and A. Zhang, 'Benchmarking retrievalaugmented generation for medicine,' arXiv preprint arXiv:2402.13178 , 2024. - · arXiv
  55. [58][58] Z. Huang, K. Xue, Y. Fan, L. Mu, R. Liu, T. Ruan, S. Zhang, and X. Zhang, 'Tool calling: Enhancing medication consultation via retrieval-augmented large language models,' arXiv preprint arXiv:2404.17897 , 2024. - · arXiv
  56. [59][59] T. Han, A. Kumar, C. Agarwal, and H. Lakkaraju, 'Towards safe and aligned large language models for medicine,' arXiv preprint arXiv:2403.03744 , 2024. - · arXiv
  57. [60][60] Z. Sun, Y. Shen, Q. Zhou, H. Zhang, Z. Chen, D. Cox, Y. Yang, and C. Gan, 'Principle-driven self-alignment of language models from scratch with minimal human supervision,' Advances in Neural Information Processing Systems , vol. 36, 2024. -
  58. [61][61] S. Manathunga and I. Hettigoda, 'Aligning large language models for clinical tasks,' arXiv preprint arXiv:2309.02884 , 2023. - · arXiv
  59. [62][62] Y. Feng, X. Xu, Y. Zhuang, and M. Zhang, 'Large language models improve alzheimer's disease diagnosis using multi-modality data,' in 2023 IEEE International Conference on Medical Artificial Intelligence , 2023, pp. 61-66. -
  60. [63][63] Z. Yuan, Q. Jin, C. Tan, Z. Zhao, H. Yuan, F. Huang, and S. Huang, 'Ramm: Retrieval-augmented biomedical visual question answering with multi-modal pre-training,' in Proceedings of the 31st ACM International Conference on Multimedia , 2023, pp. 547-556. -
  61. [64][64] J. Liu, Z. Wang, Q. Ye, D. Chong, P. Zhou, and Y. Hua, 'Qilin-med-vl: Towards chinese large vision-language model for general healthcare,' arXiv preprint arXiv:2310.17956 , 2023. - · arXiv
  62. [65][65] Y. Li, M. Du, R. Song, X. Wang, and Y. Wang, 'A survey on fairness in large language models,' arXiv preprint arXiv:2308.10149 , 2023. - · arXiv
  63. [66][66] D. H. Solomon, K. D. Allen, P. Katz, A. H. Sawalha, and E. Yelin, 'Chatgpt, et al. . . artificial intelligence, authorship, and medical publishing,' ACR Open Rheumatology , vol. 5, no. 6, p. 288, 2023. -
  64. [67][67] B. C. Das, M. H. Amini, and Y. Wu, 'Security and privacy challenges of large language models: A survey,' arXiv preprint arXiv:2402.00888 , 2024. - · arXiv
  65. [68][68] I. L. Alberts, L. Mercolli, T. Pyka, G. Prenosil, K. Shi, A. Rominger, and A. Afshar-Oromieh, 'Large language models (llm) and chatgpt: what will the impact on nuclear medicine be?' European journal of nuclear medicine and molecular imaging , vol. 50, no. 6, pp. 15491552, 2023. -
  66. [69][69] C. Wu, Z. Lin, W. Fang, and Y. Huang, 'A medical diagnostic assistant based on llm,' in China Health Information Processing Conference , 2023, pp. 135-147. -
  67. [70][70] H. Wang, S. Zhao, Z. Qiang, N. Xi, B. Qin, and T. Liu, 'Beyond direct diagnosis: Llm-based multi-specialist agent consultation for automatic diagnosis,' arXiv preprint arXiv:2401.16107 , 2024. - · arXiv
  68. [71][71] B. Mesk´ o and E. J. Topol, 'The imperative for regulatory oversight of large language models (or generative ai) in healthcare,' NPJ digital medicine , vol. 6, no. 1, p. 120, 2023. -
  69. [72][72] H. Nori, N. King, S. M. McKinney, D. Carignan, and E. Horvitz, 'Capabilities of gpt-4 on medical challenge problems,' arXiv preprint arXiv:2303.13375 , 2023. - · arXiv
  70. [73][73] B. Yan, K. Li, M. Xu, Y. Dong, Y. Zhang, Z. Ren, and X. Cheng, 'On protecting the data privacy of large language models (llms): A survey,' arXiv preprint arXiv:2403.05156 , 2024. - · arXiv
  71. [75][75] A. Act, 'Health insurance portability and accountability act of 1996,' Public law , vol. 104, p. 191, 1996. -
  72. [76][76] Z. Chu, Y. Wang, L. Li, Z. Wang, Z. Qin, and K. Ren, 'A causal explainable guardrails for large language models,' arXiv preprint arXiv:2405.04160 , 2024. - · arXiv
  73. [78][78] 'PubMed Central (PMC) Article Datasets for Text Mining,' https:// www.ncbi.nlm.nih.gov/pmc/tools/textmining/, Accessed 2023, national Center for Biotechnology Information (NCBI). -
  74. [79][79] A. J. Thirunavukarasu, D. S. J. Ting, K. Elangovan, L. Gutierrez, T. F. Tan, and D. S. W. Ting, 'Large language models in medicine,' Nature medicine , vol. 29, no. 8, pp. 1930-1940, 2023. -
  75. [80][80] J. A. Omiye, H. Gui, S. J. Rezaei, J. Zou, and R. Daneshjou, 'Large language models in medicine: The potentials and pitfalls: A narrative review,' Annals of Internal Medicine , vol. 177, no. 2, pp. 210-220, 2024. -
  76. [81][81] X. Yang, A. Chen, N. PourNejatian, H. C. Shin, K. E. Smith, C. Parisien, C. Compas, C. Martin, M. G. Flores, Y. Zhang et al. , 'Gatortron: a large clinical language model to unlock patient information from unstructured electronic health records,' arXiv preprint arXiv:2203.03540 , 2022. - · arXiv
  77. [82][82] T. H. Kung, M. Cheatham, A. Medenilla, C. Sillos, L. De Leon, C. Elepa˜ no, M. Madriaga, R. Aggabao, G. Diaz-Candido, J. Maningo et al. , 'Performance of chatgpt on usmle: potential for ai-assisted medical education using large language models,' PLoS digital health , vol. 2, no. 2, p. e0000198, 2023. -
  78. [83][83] J. Kottlors, G. Bratke, P. Rauen, C. Kabbasch, T. Persigehl, M. Schlamann, and S. Lennartz, 'Feasibility of differential diagnosis based on imaging patterns using a large language model,' Radiology , vol. 308, no. 1, p. e231167, 2023. -
  79. [84][84] Y. Huang, K. Tang, and M. Chen, 'A comprehensive survey on evaluating large language model applications in the medical industry,' arXiv preprint arXiv:2404.15777 , 2024. - · arXiv
  80. [85][85] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, 'Attention is all you need,' Advances in neural information processing systems , vol. 30, 2017. -
  81. [86][86] M. Wu, T.-T. Vu, L. Qu, G. Foster, and G. Haffari, 'Adapting large language models for document-level machine translation,' arXiv preprint arXiv:2401.06468 , 2024. - · arXiv
  82. [87][87] J. Howard and S. Ruder, 'Universal language model fine-tuning for text classification,' arXiv preprint arXiv:1801.06146 , 2018. - · arXiv
  83. [88][88] X. Qiu, T. Sun, Y. Xu, Y. Shao, N. Dai, and X. Huang, 'Pre-trained models for natural language processing: A survey,' Science China Technological Sciences , vol. 63, no. 10, pp. 1872-1897, 2020. -
  84. [89][89] K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning, 'Electra: Pretraining text encoders as discriminators rather than generators,' arXiv preprint arXiv:2003.10555 , 2020. - · arXiv
  85. [90][90] Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, 'Albert: A lite bert for self-supervised learning of language representations,' arXiv preprint arXiv:1909.11942 , 2019. - · arXiv
  86. [91][91] J. Li, A. Sun, J. Han, and C. Li, 'A survey on deep learning for named entity recognition,' IEEE transactions on knowledge and data engineering , vol. 34, no. 1, pp. 50-70, 2020. -
  87. [92][92] A. B. Abacha and P. Zweigenbaum, 'Medical entity recognition: a comparaison of semantic and statistical methods,' in Proceedings of BioNLP 2011 workshop , 2011, pp. 56-64. -
  88. [93][93] G. Zhou, J. Su, J. Zhang, and M. Zhang, 'Exploring various knowledge in relation extraction,' in Proceedings of the 43rd annual meeting of the association for computational linguistics (acl'05) , 2005, pp. 427-434. -
  89. [94][94] A. Goel, A. Gueta, O. Gilon, C. Liu, S. Erell, L. H. Nguyen, X. Hao, B. Jaber, S. Reddy, R. Kartha et al. , 'Llms accelerate annotation for medical information extraction,' in Machine Learning for Health (ML4H) . PMLR, 2023, pp. 82-100. -
  90. [95][95] J. Liu, Y . Chen, K. Liu, W. Bi, and X. Liu, 'Event extraction as machine reading comprehension,' in Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) , 2020, pp. 1641-1651. -
  91. [96][96] X. Yang, J. Bian, W. R. Hogan, and Y. Wu, 'Clinical concept extraction using transformers,' Journal of the American Medical Informatics Association , vol. 27, no. 12, pp. 1935-1942, 2020. -
  92. [97][97] J. Kim, J. Nam, S. Mo, J. Park, S.-W. Lee, M. Seo, J.-W. Ha, and J. Shin, 'Sure: Improving open-domain question answering of llms via summarized retrieval,' in The Twelfth International Conference on Learning Representations , 2023. -
  93. [98][98] L. Tang, Z. Sun, B. Idnay, J. G. Nestor, A. Soroush, P. A. Elias, Z. Xu, Y. Ding, G. Durrett, J. F. Rousseau et al. , 'Evaluating large language models on medical evidence summarization,' npj Digital Medicine , vol. 6, no. 1, p. 158, 2023. -
  94. [99][99] B. Fan, W. Fan, C. Smith et al. , 'Adverse drug event detection and extraction from open data: A deep learning approach,' Information Processing & Management , vol. 57, no. 1, p. 102131, 2020. -
  95. [100][100] P. Webster, 'Six ways large language models are changing healthcare,' Nature Medicine , vol. 29, no. 12, pp. 2969-2971, 2023. -
  96. [101][101] Y. Wang, Y. Zhao, and L. Petzold, 'Are large language models ready for healthcare? a comparative study on clinical language understanding,' in Machine Learning for Healthcare Conference . PMLR, 2023, pp. 804-823. -
  97. [102][102] Y. Zhuang, Y. Yu, K. Wang, H. Sun, and C. Zhang, 'Toolqa: A dataset for llm question answering with external tools,' Advances in Neural Information Processing Systems , vol. 36, 2024. -
  98. [103][103] S. Chen, Y. Li, S. Lu, H. Van, H. J. Aerts, G. K. Savova, and D. S. Bitterman, 'Evaluation of chatgpt family of models for biomedical reasoning and classification,' arXiv preprint arXiv:2304.02496 , 2023. - · arXiv
  99. [104][104] N. S. Tawfik and M. R. Spruit, 'Towards recognition of textual entailment in the biomedical domain,' in Natural Language Processing and Information Systems . Springer International Publishing, 2019, pp. 368-375. -
  100. [106][106] F. S. Yazi, W.-T. Vong, V. Raman, P. H. H. Then, and M. J. Lunia, 'An experimental evaluation of deep neural network model performance for the recognition of contradictory medical research claims using small and medium-sized corpora,' pp. 68-77, 2021. -
  101. [107][107] J. Yang, S. C. Han, and J. Poon, 'A survey on extraction of causal relations from natural language text,' vol. 64, no. 5, pp. 1161-1186, 2022-05-01. -
  102. [110][110] K. Lu, H. Yuan, Z. Yuan, R. Lin, J. Lin, C. Tan, C. Zhou, and J. Zhou, '# instag: Instruction tagging for analyzing supervised fine-tuning of large language models,' in The Twelfth International Conference on Learning Representations , 2023. -
  103. [111][111] S. Zhang, L. Dong, X. Li, S. Zhang, X. Sun, S. Wang, J. Li, R. Hu, T. Zhang, F. Wu et al. , 'Instruction tuning for large language models: A survey,' arXiv preprint arXiv:2308.10792 , 2023. - · arXiv
  104. [112][112] Z. Han, C. Gao, J. Liu, S. Q. Zhang et al. , 'Parameter-efficient finetuning for large models: A comprehensive survey,' arXiv preprint arXiv:2403.14608 , 2024. - · arXiv
  105. [113][113] E. J. Hu, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen et al. , 'Lora: Low-rank adaptation of large language models,' in International Conference on Learning Representations , 2021. -
  106. [114][114] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , 'Training language models to follow instructions with human feedback,' Advances in neural information processing systems , vol. 35, pp. 27 730-27 744, 2022. -
  107. [116][116] Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, and Z. Sui, 'A survey on in-context learning,' arXiv preprint arXiv:2301.00234 , 2022. - · arXiv
  108. [117][117] J.-D. Kim, T. Ohta, Y. Tateisi, and J. Tsujii, 'Genia corpus-a semantically annotated corpus for bio-textmining,' Bioinformatics , vol. 19, no. suppl 1, pp. i180-i182, 2003. -
  109. [118][118] J.-D. Kim, Y. Wang, T. Takagi, and A. Yonezawa, 'Overview of genia event task in bionlp shared task 2011,' in Proceedings of BioNLP shared task 2011 workshop , 2011, pp. 7-15. -
  110. [119][119] H. Gurulingappa, A. M. Rajput, A. Roberts, J. Fluck, M. HofmannApitius, and L. Toldo, 'Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports,' Journal of biomedical informatics , vol. 45, no. 5, pp. 885-892, 2012. -
  111. [120][120] S. Pradhan, N. Elhadad, B. R. South, D. Martinez, L. M. Christensen, A. Vogel, H. Suominen, W. W. Chapman, and G. K. Savova, 'Task 1: Share/clef ehealth evaluation lab 2013.' CLEF (working notes) , vol. 1179, 2013. -
  112. [121][121] J.-D. Kim, Y. Wang, and Y. Yasunori, 'The genia event extraction shared task, 2013 edition-overview,' in Proceedings of the BioNLP Shared Task 2013 Workshop , 2013, pp. 8-15. -
  113. [122][122] D. L. Mowery, S. Velupillai, B. R. South, L. Christensen, D. Martinez, L. Kelly, L. Goeuriot, N. Elhadad, S. Pradhan, G. Savova et al. , 'Task 2: Share/clef ehealth evaluation lab 2014,' in Proceedings of CLEF 2014 , 2014. -
  114. [123][123] Z. Sun, J. Li, G. Pergola, B. C. Wallace, B. John, N. Greene, J. Kim, and Y. He, 'Phee: A dataset for pharmacovigilance event extraction from text,' arXiv preprint arXiv:2210.12560 , 2022. - · arXiv
  115. [124][124] A. Pampari, P. Raghavan, J. Liang, and J. Peng, 'Emrqa: A large corpus for question answering on electronic medical records,' in 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 , 2018, pp. 2357-2368. -
  116. [125][125] Z. Wei, Q. Liu, B. Peng, H. Tou, T. Chen, X.-J. Huang, K.-F. Wong, and X. Dai, 'Task-oriented dialogue system for automatic diagnosis,' in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , 2018, pp. 201207. -
  117. [126][126] A. B. Abacha, Y. Mrabet, M. Sharp, T. R. Goodwin, S. E. Shooshan, and D. Demner-Fushman, 'Bridging the gap between consumers' medication questions and trusted answers.' in MedInfo , 2019, pp. 2529. -
  118. [127][127] A. Ben Abacha and D. Demner-Fushman, 'A question-entailment approach to question answering,' BMC Bioinform. , vol. 20, no. 1, pp. 511:1-511:23, 2019. -
  119. [128][128] J. He, M. Fu, and M. Tu, 'Applying deep matching networks to chinese medical question answering: a study and a dataset,' BMC medical informatics and decision making , vol. 19, no. 2, pp. 91-100, 2019. -
  120. [129][129] Q. Liu, S. Jiang, Y. Wang, and S. Li, 'Liveqa: A question answering dataset over sports live,' in Chinese Computational Linguistics: 19th China National Conference, CCL 2020, Hainan, China, October 30November 1, 2020, Proceedings 19 . Springer, 2020, pp. 316-328. -
  121. [131][131] M. Savery, A. B. Abacha, S. Gayen, and D. Demner-Fushman, 'Question-driven summarization of answers to consumer health questions,' Scientific Data , vol. 7, no. 1, p. 322, 2020. -
  122. [132][132] L. L. Wang, K. Lo, Y. Chandrasekhar, R. Reas, J. Yang, D. Burdick, D. Eide, K. Funk, Y. Katsis, R. Kinney et al. , 'Cord-19: The covid-19 open research dataset,' ArXiv , 2020. -
  123. [133][133] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, 'Measuring massive multitask language understanding,' Proceedings of the International Conference on Learning Representations (ICLR) , 2021. -
  124. [134][134] F. Xia, B. Li, Y. Weng, S. He, K. Liu, B. Sun, S. Li, and J. Zhao, 'MedConQA: Medical conversational question answering system based on knowledge graphs,' in Proceedings of the The 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , 2022, pp. 148-158. -
  125. [135][135] Y. Li, Z. Li, K. Zhang, R. Dan, S. Jiang, and Y. Zhang, 'Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge,' Cureus , vol. 15, no. 6, 2023. -
  126. [136][136] J. Li, X. Wang, X. Wu, Z. Zhang, X. Xu, J. Fu, P. Tiwari, X. Wan, and B. Wang, 'Huatuo-26m, a large-scale chinese medical qa dataset,' arXiv preprint arXiv:2305.01526 , 2023. - · arXiv
  127. [137][137] A. E. Johnson, T. J. Pollard, L. Shen, L.-w. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. Anthony Celi, and R. G. Mark, 'Mimic-iii, a freely accessible critical care database,' Scientific data , vol. 3, no. 1, pp. 1-9, 2016. -
  128. [138][138] A. B. Abacha and D. Demner-Fushman, 'On the summarization of consumer health questions,' in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , 2019, pp. 2228-2234. -
  129. [139][139] S. Sotudeh, N. Goharian, and Z. Young, 'Mentsum: A resource for exploring summarization of mental health online posts,' in Proceedings of the thirteenth language resources and evaluation conference , 2022, pp. 2682-2692. -
  130. [140][140] M. Agrawal, S. Hegselmann, H. Lang, Y. Kim, and D. Sontag, 'Large language models are few-shot clinical information extractors,' in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , 2022, pp. 1998-2022. -
  131. [141][141] B. Nye, J. J. Li, R. Patel, Y. Yang, I. J. Marshall, A. Nenkova, and B. C. Wallace, 'A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature,' in Proceedings of the conference. Association for Computational Linguistics. Meeting , vol. 2018. NIH Public Access, 2018, p. 197. -
  132. [142][142] S. Moon, S. Pakhomov, N. Liu, J. O. Ryan, and G. B. Melton, 'A sense inventory for clinical abbreviations and acronyms created using clinical notes and medical dictionary resources,' Journal of the American Medical Informatics Association , vol. 21, no. 2, pp. 299-307, 2014. -
  133. [143][143] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, 'Biobert: a pre-trained biomedical language representation model for biomedical text mining,' Bioinformatics , vol. 36, no. 4, pp. 1234-1240, 2020. -
  134. [144][144] K. Huang, J. Altosaar, and R. Ranganath, 'Clinicalbert: Modeling clinical notes and predicting hospital readmission,' arXiv preprint arXiv:1904.05342 , 2019. - · arXiv
  135. [145][145] H.-C. Shin, Y. Zhang, E. Bakhturina, R. Puri, M. Patwary, M. Shoeybi, and R. Mani, 'Biomegatron: larger biomedical domain language model,' in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2020, pp. 4700-4706. -
  136. [148][148] V. Li´ evin, C. E. Hother, A. G. Motzfeldt, and O. Winther, 'Can large language models reason about medical questions?' Patterns , vol. 5, no. 3, 2024. -
  137. [149][149] R. Taylor, M. Kardas, G. Cucurull, T. Scialom, A. Hartshorn, E. Saravia, A. Poulton, V. Kerkez, and R. Stojnic, 'Galactica: A large language model for science,' arXiv preprint arXiv:2211.09085 , 2022. - · arXiv
  138. [150][150] Z. Liu, Y. Huang, X. Yu, L. Zhang, Z. Wu, C. Cao, H. Dai, L. Zhao, Y. Li, P. Shu et al. , 'Deid-gpt: Zero-shot medical text de-identification by gpt-4,' arXiv preprint arXiv:2303.11032 , 2023. - · arXiv
  139. [151][151] L. Yunxiang, L. Zihan, Z. Kai, D. Ruilong, and Z. You, 'Chatdoctor: A medical chat model fine-tuned on llama model using medical domain knowledge,' arXiv preprint arXiv:2303.14070 , 2023. - · arXiv
  140. [152][152] H. Xiong, S. Wang, Y. Zhu, Z. Zhao, Y. Liu, L. Huang, Q. Wang, and D. Shen, 'Doctorglm: Fine-tuning your chinese doctor is not a herculean task,' arXiv preprint arXiv:2304.01097 , 2023. - · arXiv
  141. [153][153] H. Wang, C. Liu, N. Xi, Z. Qiang, S. Zhao, B. Qin, and T. Liu, 'Huatuo: Tuning llama model with chinese medical knowledge,' arXiv preprint arXiv:2304.06975 , 2023. - · arXiv
  142. [154][154] C. Wu, X. Zhang, Y. Zhang, Y. Wang, and W. Xie, 'Pmc-llama: Further finetuning llama on medical papers,' arXiv preprint arXiv:2304.14454 , 2023. - · arXiv
  143. [157][157] K. Singhal, T. Tu, J. Gottweis, R. Sayres, E. Wulczyn, L. Hou, K. Clark, S. Pfohl, H. Cole-Lewis, D. Neal et al. , 'Towards expertlevel medical question answering with large language models,' arXiv preprint arXiv:2305.09617 , 2023. - · arXiv
  144. [158][158] C. Peng, X. Yang, A. Chen, K. E. Smith, N. PourNejatian, A. B. Costa, C. Martin, M. G. Flores, Y. Zhang, T. Magoc et al. , 'A study of generative large language model for medical research and healthcare,' NPJ Digital Medicine , vol. 6, no. 1, p. 210, 2023. -
  145. [159][159] H. Zhang, J. Chen, F. Jiang, F. Yu, Z. Chen, J. Li, G. Chen, X. Wu, Z. Zhang, Q. Xiao et al. , 'Huatuogpt, towards taming language model to be a doctor,' arXiv preprint arXiv:2305.15075 , 2023. - · arXiv
  146. [160][160] J. Zhou, X. Chen, and X. Gao, 'Path to medical agi: Unify domainspecific medical llms with the lowest cost,' medRxiv , pp. 2023-06, 2023. -
  147. [161][161] W. Gao, Z. Deng, Z. Niu, F. Rong, C. Chen, Z. Gong, W. Zhang, D. Xiao, F. Li, Z. Cao et al. , 'Ophglm: Training an ophthalmology large language-and-vision assistant based on instructions and dialogue,' arXiv preprint arXiv:2306.12174 , 2023. - · arXiv
  148. [162][162] Y. Chen, X. Xing, J. Lin, H. Zheng, Z. Wang, Q. Liu, and X. Xu, 'Soulchat: Improving llms' empathy, listening, and comfort abilities through fine-tuning with multi-turn empathy conversations,' in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023, pp. 1170-1183. -
  149. [163][163] M. Moor, Q. Huang, S. Wu, M. Yasunaga, Y. Dalmia, J. Leskovec, C. Zakka, E. P. Reis, and P. Rajpurkar, 'Med-flamingo: a multimodal medical few-shot learner,' in Machine Learning for Health (ML4H) . PMLR, 2023, pp. 353-367. -
  150. [170][170] L. Luo, J. Ning, Y. Zhao, Z. Wang, Z. Ding, P. Chen, W. Fu, Q. Han, G. Xu, Y. Qiu, D. Pan, J. Li, H. Li, W. Feng, S. Tu, Y. Liu, Z. Yang, J. Wang, Y. Sun, and H. Lin, 'Taiyi: A bilingual fine-tuned large language model for diverse biomedical tasks,' Journal of the American Medical Informatics Association , 2024. -
  151. [171][171] S. Yang, H. Zhao, S. Zhu, G. Zhou, H. Xu, Y. Jia, and H. Zan, 'Zhongjing: Enhancing the chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 17, 2024, pp. 19 368-19 376. -
  152. [173][173] M. Jin, Q. Yu, C. Zhang, D. Shu, S. Zhu, M. Du, Y. Zhang, and Y. Meng, 'Health-llm: Personalized retrieval-augmented disease prediction model,' arXiv preprint arXiv:2402.00746 , 2024. - · arXiv
  153. [174][174] J. J. Norcini, L. L. Blank, F. D. Duffy, and G. S. Fortna, 'The mini-cex: a method for assessing clinical skills,' Annals of internal medicine , vol. 138, no. 6, pp. 476-481, 2003. -
  154. [175][175] J. R. Kogan, L. M. Bellini, and J. A. Shea, 'Implementation of the minicex to evaluate medical students' clinical skills,' Academic Medicine , vol. 77, no. 11, pp. 1156-1157, 2002. -
  155. [176][176] C.-H. Chiang and H.-y. Lee, 'Can large language models be an alternative to human evaluations?' arXiv preprint arXiv:2305.01937 , 2023. - · arXiv
  156. [177][177] C. Mattingly, 'What is clinical reasoning?' The American Journal of Occupational Therapy , vol. 45, no. 11, pp. 979-986, 1991. -
  157. [178][178] Z. Chu, L. Zhang, Y. Sun, S. Xue, Z. Wang, Z. Qin, and K. Ren, 'Sora detector: A unified hallucination detection for large text-to-video models,' arXiv preprint arXiv:2405.04180 , 2024. - · arXiv
  158. [179][179] Y. Wang, Z. Chu, X. Ouyang, S. Wang, H. Hao, Y. Shen, J. Gu, S. Xue, J. Zhang, Q. Cui et al. , 'Llmrg: Improving recommendations through large language model reasoning graphs,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 17, 2024, pp. 19 18919 196. -
  159. [180][180] Y. Wang, Z. Chu, X. Ouyang, S. Wang, H. Hao, Y. Shen, J. Gu, S. Xue, J. Y. Zhang, Q. Cui et al. , 'Enhancing recommender systems with large language model reasoning graphs,' arXiv preprint arXiv:2308.10835 , 2023. - · arXiv
  160. [181][181] Z. Chu, Y. Wang, Q. Cui, L. Li, W. Chen, S. Li, Z. Qin, and K. Ren, 'Llm-guided multi-view hypergraph learning for human-centric explainable recommendation,' arXiv preprint arXiv:2401.08217 , 2024. - · arXiv
  161. [182][182] H.-S. Sheu, Z. Chu, D. Qi, and S. Li, 'Knowledge-guided article embedding refinement for session-based news recommendation,' IEEE Transactions on Neural Networks and Learning Systems , vol. 33, no. 12, pp. 7921-7927, 2021. -
  162. [183][183] S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang, and X. Wu, 'Unifying large language models and knowledge graphs: A roadmap,' IEEE Transactions on Knowledge and Data Engineering , 2024. -
  163. [184][184] J. Baek, A. F. Aji, and A. Saffari, 'Knowledge-augmented language model prompting for zero-shot knowledge graph question answering,' arXiv preprint arXiv:2306.04136 , 2023. - · arXiv
  164. [185][185] Y. Tian, H. Song, Z. Wang, H. Wang, Z. Hu, F. Wang, N. V. Chawla, and P. Xu, 'Graph neural prompting with large language models,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 17, 2024, pp. 19 080-19 088. -
  165. [186][186] W. Wei, X. Ren, J. Tang, Q. Wang, L. Su, S. Cheng, J. Wang, D. Yin, and C. Huang, 'Llmrec: Large language models with graph augmentation for recommendation,' in Proceedings of the 17th ACM International Conference on Web Search and Data Mining , 2024, pp. 806-815. -
  166. [187][187] Y. Wu, N. Hu, G. Qi, S. Bi, J. Ren, A. Xie, and W. Song, 'Retrieverewrite-answer: A kg-to-text enhanced llms framework for knowledge graph question answering,' arXiv preprint arXiv:2309.11206 , 2023. - · arXiv
  167. [188][188] X. Guan, Y. Liu, H. Lin, Y. Lu, B. He, X. Han, and L. Sun, 'Mitigating large language model hallucinations via autonomous knowledge graphbased retrofitting,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 16, 2024, pp. 18 126-18 134. -
  168. [189][189] Q. Li, X. Yang, H. Wang, Q. Wang, L. Liu, J. Wang, Y. Zhang, M. Chu, S. Hu, Y. Chen et al. , 'From beginner to expert: Modeling medical knowledge into general llms,' arXiv preprint arXiv:2312.01040 , 2023. - · arXiv
  169. [190][190] L. Yang, H. Chen, Z. Li, X. Ding, and X. Wu, 'Chatgpt is not enough: Enhancing large language models with knowledge graphs for factaware language modeling,' arXiv preprint arXiv:2306.11489 , 2023. - · arXiv
  170. [194][194] Y. Talebirad and A. Nadiri, 'Multi-agent collaboration: Harnessing the power of intelligent llm agents,' arXiv preprint arXiv:2306.03314 , 2023. - · arXiv
  171. [195][195] Z. Chu, Y. Wang, F. Zhu, L. Yu, L. Li, and J. Gu, 'Professional agentsevolving large language models into autonomous experts with humanlevel competencies,' arXiv preprint arXiv:2402.03628 , 2024. - · arXiv
  172. [196][196] Y. Guan, D. Wang, Z. Chu, S. Wang, F. Ni, R. Song, L. Li, J. Gu, and C. Zhuang, 'Intelligent virtual assistants with llm-based process automation,' arXiv preprint arXiv:2312.06677 , 2023. - · arXiv
  173. [197][197] Y. Zhu, S. Qiao, Y. Ou, S. Deng, N. Zhang, S. Lyu, Y. Shen, L. Liang, J. Gu, and H. Chen, 'Knowagent: Knowledge-augmented planning for llm-based agents,' arXiv preprint arXiv:2403.03101 , 2024. - · arXiv
  174. [198][198] Y. Hou, H. Tamoto, and H. Miyashita, '' my agent understands me better': Integrating dynamic human-like memory recall and consolidation in llm-based agents,' arXiv preprint arXiv:2404.00573 , 2024. - · arXiv
  175. [199][199] K. Mei, Z. Li, S. Xu, R. Ye, Y. Ge, and Y. Zhang, 'Llm agent operating system,' arXiv preprint arXiv:2403.16971 , 2024. - · arXiv
  176. [200][200] N. Mehandru, B. Y. Miao, E. R. Almaraz, M. Sushil, A. J. Butte, and A. Alaa, 'Evaluating large language models as agents in the clinic,' npj Digital Medicine , vol. 7, no. 1, p. 84, 2024. -
  177. [201][201] X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang et al. , 'Agentbench: Evaluating llms as agents,' arXiv preprint arXiv:2308.03688 , 2023. - · arXiv
  178. [202][202] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, 'React: Synergizing reasoning and acting in language models,' arXiv preprint arXiv:2210.03629 , 2022. - · arXiv
  179. [203][203] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , 'Gpt-4 technical report,' arXiv preprint arXiv:2303.08774 , 2023. - · arXiv
  180. [204][204] D. Zhou, N. Sch¨ arli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le et al. , 'Least-to-most prompting enables complex reasoning in large language models,' arXiv preprint arXiv:2205.10625 , 2022. - · arXiv
  181. [205][205] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, and H. Wang, 'Retrieval-augmented generation for large language models: A survey,' arXiv preprint arXiv:2312.10997 , 2023. - · arXiv
  182. [206][206] J. Ramos et al. , 'Using tf-idf to determine word relevance in document queries,' in Proceedings of the first instructional conference on machine learning , vol. 242, no. 1, 2003, pp. 29-48. -
  183. [207][207] A. Trotman, A. Puurula, and B. Burgess, 'Improvements to bm25 and language models examined,' in Proceedings of the 19th Australasian Document Computing Symposium , 2014, pp. 58-65. -
  184. [208][208] G. Izacard, P. Lewis, M. Lomeli, L. Hosseini, F. Petroni, T. Schick, J. Dwivedi-Yu, A. Joulin, S. Riedel, and E. Grave, 'Atlas: Fewshot learning with retrieval augmented language models,' Journal of Machine Learning Research , vol. 24, no. 251, pp. 1-43, 2023. -
  185. [209][209] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. K¨ uttler, M. Lewis, W.-t. Yih, T. Rockt¨ aschel et al. , 'Retrievalaugmented generation for knowledge-intensive nlp tasks,' Advances in Neural Information Processing Systems , vol. 33, pp. 9459-9474, 2020. -
  186. [210][210] D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu et al. , 'Palm-e: An embodied multimodal language model,' arXiv preprint arXiv:2303.03378 , 2023. - · arXiv
  187. [211][211] H. Liu, C. Li, Q. Wu, and Y. J. Lee, 'Visual instruction tuning,' Advances in neural information processing systems , vol. 36, 2024. -
  188. [212][212] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , 'Learning transferable visual models from natural language supervision,' in International conference on machine learning . PMLR, 2021, pp. 8748-8763. -
  189. [213][213] W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez et al. , 'Vicuna: An opensource chatbot impressing gpt-4 with 90%* chatgpt quality,' See https://vicuna. lmsys. org (accessed 14 April 2023) , vol. 2, no. 3, p. 6, 2023. -
  190. [214][214] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ ar, and C. L. Zitnick, 'Microsoft coco: Common objects in context,' in Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 , 2014, pp. 740-755. -
  191. [215][215] Q. Ye, H. Xu, G. Xu, J. Ye, M. Yan, Y. Zhou, J. Wang, A. Hu, P. Shi, Y. Shi et al. , 'mplug-owl: Modularization empowers large language models with multimodality,' arXiv preprint arXiv:2304.14178 , 2023. - · arXiv
  192. [216][216] I. Hartsock and G. Rasool, 'Vision-language models for medical report generation and visual question answering: A review,' arXiv preprint arXiv:2403.02469 , 2024. - · arXiv
  193. [217][217] Z. Chu, M. Hu, Q. Cui, L. Li, and S. Li, 'Task-driven causal feature distillation: Towards trustworthy risk prediction,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 10, 2024, pp. 11 642-11 650. -
  194. [218][218] B. Li, T. Meng, X. Shi, J. Zhai, and T. Ruan, 'Meddm: Llm-executable clinical guidance tree for clinical decision-making,' arXiv preprint arXiv:2312.02441 , 2023. - · arXiv
  195. [219][219] Z. Chu, H. Hao, X. Ouyang, S. Wang, Y. Wang, Y. Shen, J. Gu, Q. Cui, L. Li, S. Xue et al. , 'Leveraging large language models for pre-trained recommender systems,' arXiv preprint arXiv:2308.10837 , 2023. - · arXiv
  196. [220][220] Z. Chu, S. L. Rathbun, and S. Li, 'Matching in selective and balanced representation space for treatment effects estimation,' in Proceedings of the 29th ACM International Conference on Information & Knowledge Management , 2020, pp. 205-214. -
  197. [221][221] S. Wang, Z. Zhao, X. Ouyang, Q. Wang, and D. Shen, 'Chatcad: Interactive computer-aided diagnosis on medical image using large language models,' arXiv preprint arXiv:2302.07257 , 2023. - · arXiv
  198. [222][222] V. Sorin, Y. Barash, E. Konen, and E. Klang, 'Large language models for oncological applications,' Journal of Cancer Research and Clinical Oncology , vol. 149, no. 11, pp. 9505-9508, 2023. -
  199. [223][223] Z. Chen, L. Luo, Y. Bie, and H. Chen, 'Dia-llama: Towards large language model-driven ct report generation,' arXiv preprint arXiv:2403.16386 , 2024. - · arXiv
  200. [224][224] C. Ma, Z. Wu, J. Wang, S. Xu, Y. Wei, Z. Liu, X. Jiang, L. Guo, X. Cai, S. Zhang et al. , 'Impressiongpt: an iterative optimizing framework for radiology report summarization with chatgpt,' arXiv preprint arXiv:2304.08448 , 2023. - · arXiv

200 of 237 shown

Cited by

Sorted by citations

Related

Authors
Liu, L., Yang, X., Lei, J., Shen, Y., Wang, J., Wei, P., Chu, Z., Qin, Z., & Ren, K.
Affiliations
  • Ant Group (China) (CN)
  • Zhejiang University (CN)
ORCID
Venue
IEEE Transactions on Knowledge and Data Engineering
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2026-11
Volume
38
Issue
11
Pages
7189-7210
ISSN
  • 1041-4347 (ISSN-L)
  • 1558-2191
  • 2326-3865
DOI
10.1109/tkde.2026.3709941
OpenAlex
W4399454054
Cited by
31
References
0
Citations by year
2026: 9 · 2025: 20 · 2024: 2
FWCI
17.70
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Knowledge and Data Engineering (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
Ant Group Postdoctoral Programme
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
medical large language models, artificial intelligence in healthcare, clinical decision support, medical education, patient communication, fairness, accountability, and transparency, privacy, robustness, regulatory framework, trustworthiness, ethical considerations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review