The DOI matches its Crossref record; 136 of the 136 references it cites were found in a registry or an index.
Opportunities and challenges for ChatGPT and large language models in biomedicine and health
Open accessAbstract
ChatGPT has drawn considerable attention from both the general public and domain experts with its remarkable text generation capabilities. This has subsequently led to the emergence of diverse applications in the field of biomedicine and health. In this work, we examine the diverse applications of large language models (LLMs), such as ChatGPT, in biomedicine and health. Specifically, we explore the areas of biomedical information retrieval, question answering, medical text summarization, information extraction and medical education and investigate whether LLMs possess the transformative power to revolutionize these tasks or whether the distinct complexities of biomedical domain presents unique challenges. Following an extensive literature survey, we find that significant advances have been made in the field of text generation tasks, surpassing the previous state-of-the-art methods. For other applications, the advances have been modest. Overall, LLMs have not yet revolutionized biomedicine, but recent rapid progress indicates that such methods hold great potential to provide valuable means for accelerating discovery and improving health. We also find that the use of LLMs, like ChatGPT, in the fields of biomedicine and health entails various risks and challenges, including fabricated information in its generated responses, as well as legal and privacy concerns associated with sensitive patient data. We believe this survey can provide a comprehensive and timely overview to biomedical researchers and healthcare practitioners on the opportunities and challenges associated with using ChatGPT and other LLMs for transforming biomedicine and health.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Prompt engineering improves chatbot performance by tailoring LLM output to be focused and helpful for a specific task.162,163”
From Large Language Models for Chatbot Health Advice Studies: A Systematic Review (2025) · cites this paper as [162]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2_v1 · Cited by 8,090
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“Unfair and biased model in the biomedical and health domain can lead to detrimental outcomes and affect the quality of treatment a patient receives [115-117].”
BackgroundThe PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2 · Cited by 5,177
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1×
“To address the hallucination issue [65] in medical QA systems, one of the current solutions is retrieval augmentation, which refers to the approach of combining LLMs with a search system, such as the New Bing for the general domain and Almanac [66] in the clinical domain.”
MethodChain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,451
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1×
“Another issue is copyright – in many jurisdictions, machine-generated material may not receive a copyright [120, 131, 136], which poses an obvious problem for journals.”
BackgroundBERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Proceedings of the 13th International Conference on Neural Information Processing Systems
Leen, T. K., Dietterich, T. G., & Tresp, V. (2000) · Cited by 2,190
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · Scientific Data · doi:10.1038/s41597-019-0322-0 · Cited by 1,816
Toward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734
Hidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms
Vyas, D. A., Eisenstein, L. G., & Jones, D. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmms2004740 · Cited by 1,731
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
Self-citationEmergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264
Dave, T., Athaluri, S. A., & Singh, S. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1169595 · Cited by 1,204
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00819-6 · Cited by 1,099
Legal and Ethical Consideration in Artificial Intelligence in Healthcare: Who Takes Responsibility?
Naik, N., Hameed, B. M. Z., Shetty, D. K., Swain, D., Shah, M., Paul, R., Aggarwal, K., Ibrahim, S., Patil, V., Smriti, K., et al. (2022) · Frontiers in Surgery · doi:10.3389/fsurg.2022.862322 · Cited by 931
ChatGPT listed as author on research papers: many scientists disapprove
Stokel-Walker, C. (2023) · Nature · doi:10.1038/d41586-023-00107-z · Cited by 868
NCBI disease corpus: A resource for disease name recognition and concept normalization
Islamaj, R., Leaman, R., & Lu, Z. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.12.006 · Cited by 857
Self-citationEysenbach, G. (2023) · JMIR Medical Education · doi:10.2196/46885 · Cited by 840
GPTScore: Evaluate as You Desire
Fu, J., Ng, S.-K., Jiang, Z., & Liu, P. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.365 · Cited by 791
Li, Y., Li, Z., Zhang, K., Dan, R., Jiang, S. B., & Zhang, Y. (2023) · Cureus · doi:10.7759/cureus.40895 · Cited by 784
ChatGPT: the future of discharge summaries?
Patel, S. B., & Lam, K. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00021-3 · Cited by 728
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · Applied Sciences · doi:10.3390/app11146421 · Cited by 718
Murray, G., Renals, S. J., Carletta, J., & Moore, J. D. (2005) · Meeting of the Association for Computational Linguistics · Cited by 716
Harnessing the power of llms in practice: a survey on chatGPT and beyond
Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., & Hu, X. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3649506 · Cited by 644
ChatGPT - Reshaping medical education and clinical management
Khan, R. A., Jawaid, M., Khan, A. R., & Sajjad, M. (2023) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.39.2.7653 · Cited by 628
Tsatsaronis, G., Balikas, G., Malakasiotis, P., Partalas, I., Zschunke, M., Alvers, M. R., Weissenborn, D., Krithara, A., Petridis, S., Polychronopoulos, D., et al. (2015) · BMC Bioinformatics · doi:10.1186/s12859-015-0564-6 · Cited by 598
PubMed and beyond: a survey of web tools for searching biomedical literature
Lu, Z. (2011) · Database · doi:10.1093/database/baq036 · Cited by 596
Self-citationOpen artificial intelligence platforms in nursing education: Tools for academic progress or abuse?
O’Connor, S., & ChatGPT (2023) · Nurse Education in Practice · doi:10.1016/j.nepr.2022.103537 · Cited by 585
González-Gallardo, C. E., Montes, A., Sierra, G., Núñez-Juárez, J. A., Salinas-López, A. J., & Ek, J. (2015) · conference · Cited by 559
ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports
Jeblick, K., Schachtner, B., Dexl, J., Mittermeier, A., Stüber, A. T., Topalis, J., Weber, T., Wesp, P., Sabel, B. O., Ricke, J., et al. (2023) · European Radiology · doi:10.1007/s00330-023-10213-1 · Cited by 549
Capabilities of GPT-4 on Medical Challenge Problems
Nori, H., King, N., McKinney, S. M., Carignan, D., & Horvitz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13375 · Cited by 509
MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training DataPreprint
Han, T., Adams, L. C., Papaioannou, J.-M., Grundmann, P., Oberhauser, T., Figueroa, A., Löser, A., Truhn, D., & Bressem, K. K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.08247 · Cited by 505
The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492
Overview of BioCreative II gene mention recognition
Smith, L. H., Tanabe, L. K., Ando, R. J. N., Kuo, C.-J., Chung, I., Hsu, C., Lin, Y.-S., Klinger, R., Friedrich, C. M., Ganchev, K., et al. (2008) · Genome biology · doi:10.1186/gb-2008-9-s2-s2 · Cited by 486
The DDI corpus: An annotated corpus with pharmacological substances and drug–drug interactions
Herrero-Zazo, M., Segura-Bédmar, I., Martı́nez, P., & Declerck, T. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.07.011 · Cited by 479
ChatGPT in healthcare: A taxonomy and systematic review
Li, J., Dada, A., Puladi, B., Kleesiek, J., & Egger, J. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108013 · Cited by 452
Almanac — Retrieval-Augmented Language Models for Clinical Medicine
Zakka, C., Shad, R., Chaurasia, A., Dalal, A. R., Kim, J. L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., et al. (2024) · NEJM AI · doi:10.1056/aioa2300068 · Cited by 424
Evaluating large language models on medical evidence summarization
Tang, L., Sun, Z., Idnay, B., Nestor, J. G., Soroush, A., Elias, P. A., Xu, Z., Ding, Y., Durrett, G., Rousseau, J. F., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00896-7 · Cited by 423
Answering Physicians' Clinical Questions: Obstacles and Potential Solutions
Ely, J., Osheroff, J. A., Chambliss, M. L., Ebell, M. H., & Rosenbaum, M. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1608 · Cited by 391
Can large language models reason about medical questions?
Liévin, V., Hother, C. E., Motzfeldt, A. G., & Winther, O. (2024) · Patterns · doi:10.1016/j.patter.2024.100943 · Cited by 343 · cited 1×
“…Encoder Representations from Transformers) [17] and its variants, decoder-only LMs such as the GPT (Generative Pre-trained Transformer) family [18-20] and encoder-decoder LMs such as T5 (Text-to-Text Transfer Transformer) [21] and BART (Bidirectional and AutoRegressive Transformers) [22].”
BackgroundTowards Expert-Level Medical Question Answering with Large Language Models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.09617 · Cited by 338
Matching patients to clinical trials with large language models
Jin, Q., Wang, Z., Floudas, C. S., Chen, F., Gong, C., Bracken-Clarke, D., Xue, E., Yang, Y., Sun, J., & Lu, Z. (2024) · Nature Communications · doi:10.1038/s41467-024-53081-z · Cited by 287
Self-citationCan ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?
Wang, S., Scells, H., Koopman, B., & Zuccon, G. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3591703 · Cited by 275
MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs
Johnson, A. E. W., Pollard, T. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Peng, Y., Lu, Z., Mark, R. G., Berkowitz, S. J., & Horng, S. (2019) · arXiv · doi:10.48550/arxiv.1901.07042 · Cited by 269
Self-citationBravo, Á., Piñero, J., Queralt-Rosiñach, N., Rautschka, M., & Furlong, L. I. (2015) · BMC Bioinformatics · doi:10.1186/s12859-015-0472-9 · Cited by 266
PubMed related articles: a probabilistic topic-based model for content similarity
Lin, J., & Wilbur, W. J. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-423 · Cited by 243
Automated methods for the summarization of electronic health records
Pivovarov, R., & Elhadad, N. (2015) · J. Am. Medical Informatics Assoc. · doi:10.1093/jamia/ocv032 · Cited by 238
Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model
Bengio, Y., & Senécal, J.-S. (2008) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.912312 · Cited by 223
Chen, Q., Hu, Y., Peng, X., Xie, Q., Jin, Q., Gilson, A., Singer, M. B., Ai, X., Lai, P.-T., Wang, Z., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-56989-2 · Cited by 219 · cited 1×
“Chen et al. [105] performed a pilot study to establish the baselines of using GPT-3.”
BackgroundSelf-citationEmbracing Large Language Models for Medical Applications: Opportunities and Challenges
Karabacak, M., & Margetis, K. D. (2023) · Cureus · doi:10.7759/cureus.39305 · Cited by 217
Rogers, A., Boyd-Graber, J., & Okazaki, N. (2023) · doi:10.18653/v1/2023.acl-short · Cited by 208
Rapamycin in the context of Pascal's Wager: generative pre-trained transformer perspective
Generative Pre-trained Transformer, C., & Zhavoronkov, A. (2022) · Oncoscience · doi:10.18632/oncoscience.571 · Cited by 203
Wei, C.-H., Peng, Y., Leaman, R., Davis, A. P., Mattingly, C. J., Li, J., Wiegers, T. C., & Lu, Z. (2016) · Database · doi:10.1093/database/baw032 · Cited by 198
Self-citationYang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C. B., Martin, C. E., Flores, M. G., Zhang, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.03540 · Cited by 190
(2024) · doi:10.1145/3704435 · Cited by 184
Tool Learning with Foundation Models
Qin, Y., Hu, S., Lin, Y., Chen, W., Ding, N., Cui, G., Zeng, Z., Huang, Y., Xiao, C., Han, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.08354 · Cited by 184
Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again
Gutiérrez, B. J., McNeal, N., Washington, C., Chen, Y., Li, L., Sun, H., & Su, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.329 · Cited by 182
Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society
(2022) · doi:10.1145/3514094 · Cited by 182
Jin, Q., Yang, Y., Chen, Q., & Lu, Z. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae075 · Cited by 176
Self-citationBioMegatron: Larger Biomedical Domain Language Model
Shin, H.-C., Zhang, Y., Bakhturina, E., Puri, R., Patwary, M. A., Shoeybi, M., & Mani, R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.379 · Cited by 175
Caufield, J. H., Hegde, H., Emonet, V., Harris, N. L., Joachimiak, M. P., Matentzoglu, N., Kim, H., Moxon, S. A. T., Reese, J., Haendel, M. A., et al. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae104 · Cited by 172
Proceedings of the BioNLP Shared Task 2011 Workshop
Tsujii, J., & Association for Computing Machinery-Digital Library (2011) · Cited by 160
Interactive computer-aided diagnosis on medical image using large language models
Wang, S., Zhao, Z., Ouyang, X., Liu, T., Wang, Q., & Shen, D. G. (2024) · Communications Engineering · doi:10.1038/s44172-024-00271-8 · Cited by 157
PAL: Program-aided Language Models
Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., & Neubig, G. (2022) · arXiv · doi:10.48550/arxiv.2211.10435 · Cited by 155
Self-citationDeep Bidirectional Language-Knowledge Graph Pretraining
Yasunaga, M., Bosselut, A., Ren, H., Zhang, X., Manning, C. D., Liang, P., & Leskovec, J. (2022) · neural information processing systems · doi:10.52202/068431-2704 · Cited by 154
Biomedical Question Answering: A Survey of Approaches and Challenges
Jin, Q., Yuan, Z. P., Xiong, G., Yu, Q., Ying, H., Tan, C., Chen, M., Huang, S., Liu, X., & Yu, S. (2022) · ACM Computing Surveys · doi:10.1145/3490238 · Cited by 145
Self-citationEvaluating Verifiability in Generative Search Engines
Liu, N., Zhang, T., & Liang, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.467 · Cited by 143
Evaluating Verifiability in Generative Search Engines
Liu, N. F., Zhang, T., & Liang, P. (2023) · arXiv · Cited by 143 · cited 2×
“However, such systems are still not free from errors [54] and there is a need for more systematic evaluations [52].”
BackgroundAn Iterative Optimizing Framework for Radiology Report Summarization With ChatGPT
Ma, C., Wu, Z., Wang, J., Xu, S., Wei, Y., Liu, Z., Zeng, F., Jiang, X., Guo, L., Cai, X., et al. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3364586 · Cited by 139
Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models
Tamkin, A., Brundage, M., Clark, J., & Ganguli, D. (2021) · arXiv · doi:10.48550/arxiv.2102.02503 · Cited by 135
Can an artificial intelligence chatbot be the author of a scholarly article?
Lee, J. Y. (2023) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2022.20.6 · Cited by 131
Automatic semantic classification of scientific literature according to the hallmarks of cancer
Baker, S., Silins, I., Guo, Y., Ali, I., Högberg, J., Stenius, U., & Korhonen, A. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv585 · Cited by 128
An extensive benchmark study on biomedical text generation and mining with ChatGPT
Chen, Q., Sun, H., Liu, H., Jiang, Y., Ran, T., Jin, X., Xiao, X., Lin, Z., Chen, H., & Niu, Z. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad557 · Cited by 125
Self-citationSummarization of clinical information: A conceptual model
Feblowitz, J. C., Wright, A. T., Singh, H., Samal, L., & Sittig, D. F. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.03.008 · Cited by 124
HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge
Wang, H., Liu, C., Xi, N., Qiang, Z., Zhao, S., Qin, B., & Liu, T. (2023) · arXiv · doi:10.48550/arxiv.2304.06975 · Cited by 115
ChatGPT, Bard, and large language models for biomedical research: opportunities and pitfalls
Thapa, S., & Adhikari, S. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03284-0 · Cited by 111
Can an artificial intelligence chatbot be the author of a scholarly article?
Lee, J. Y. (2023) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2023.20.6 · Cited by 108
Proceedings of the Conference on Health, Inference, and Learning
(2021) · doi:10.1145/3450439 · Cited by 93
Generating Extractive Summaries of Scientific Paradigms
Qazvinian, V., Radev, D., Mohammad, S. M., Dorr, B. J., Zajic, D. M., Whidby, M., & Moon, T. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3732 · Cited by 86
AIONER: all-in-one scheme-based biomedical named entity recognition using deep learning
Luo, L., Wei, C.-H., Lai, P., Leaman, R., Chen, Q., & Lu, Z. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad310 · Cited by 81
Self-citationDoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task
Xiong, H., Wang, S., Zhu, Y., Zhao, Z., Liu, Y., Huang, L., Wang, Q., & Shen, D. (2023) · arXiv · doi:10.48550/arxiv.2304.01097 · Cited by 81
A survey on literature based discovery approaches in biomedical domain
Gopalakrishnan, V., Jha, K., Jin, W., & Zhang, A. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103141 · Cited by 76
BioASQ-QA: A manually curated corpus for Biomedical Question Answering
Krithara, A., Nentidis, A., Bougiatiotis, K., & Παλιούρας, Γ. (2023) · Scientific Data · doi:10.1038/s41597-023-02068-4 · Cited by 75
Proceedings of the BioNLP 2018 workshop
(2018) · doi:10.18653/v1/w18-23 · Cited by 74
A Critical Review of ChatGPT as a Potential Substitute for Diabetes Educators
Sharma, S., Pajai, S., Prasad, R., Wanjari, M. B., Munjewar, P. K., Sharma, R. P., & Pathade, A. (2023) · Cureus · doi:10.7759/cureus.38380 · Cited by 66
Jin, Q., Leaman, R., & Lu, Z. (2023) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000000000166 · Cited by 64
Self-citationAI-generated text may have a role in evidence-based medicine
Peng, Y., Rousseau, J. F., Shortliffe, E. H., & Weng, C. (2023) · Nature Medicine · doi:10.1038/s41591-023-02366-9 · Cited by 64
Long Document Summarization with Top-down and Bottom-up Inference
Pang, B., Nijkamp, E., Kryściński, W., Savarese, S., Zhou, Y., & Xiong, C. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.94 · Cited by 60
Cohen, K. B., Lanfranchi, A., Choi, M., Bada, M. A., Baumgartner, W. A., Panteleyeva, N., Verspoor, K. M., Palmer, M. S., & Hunter, L. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1775-9 · Cited by 59
Benchmarking large language models for genomic knowledge with GeneTuring
Shang, X., Liao, X., Ji, Z., & Hou, W. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf492 · Cited by 53
Can GPT-3 write an academic paper on itself, with minimal human input?Preprint
Transformer, G. G. P., Thunström, A. O., & Steingrímsson, S. (2022) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 51
Bridging the Gap Between Consumers’ Medication Questions and Trusted Answers
Abacha, A. B., Mrabet, Y., Sharp, M. J., Goodwin, T. R., Shooshan, S. E., & Demner‐Fushman, D. (2019) · Studies in health technology and informatics · doi:10.3233/shti190176 · Cited by 50
Resolving anaphoras for the extraction of drug-drug interactions in pharmacological documents
Segura-Bédmar, I., Miguel, M. C., de Pablo-Sánchez, C., & Martı́nez, P. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-s2-s1 · Cited by 43
Possible Bias in Supervised Deep Learning Algorithms for CT Lung Nodule Detection and Classification
Sourlos, N., Wang, J., Nagaraj, Y., van Ooijen, P. M. A., & Vliegenthart, R. (2022) · Cancers · doi:10.3390/cancers14163867 · Cited by 40
O’Connor, S. (2023) · Nurse Education in Practice · doi:10.1016/j.nepr.2023.103572 · Cited by 40
Almanac: Retrieval-Augmented Language Models for Clinical MedicinePreprint
Hiesinger, W., Zakka, C., Chaurasia, A., Shad, R., Dalal, A. R., Kim, J. L., Moor, M., Alexander, K. M., Ashley, E. A., Boyd, J. H., et al. (2023) · Research Square · doi:10.21203/rs.3.rs-2883198/v1 · Cited by 40
Should We Acknowledge ChatGPT as an Author?
Goto, A., & Katanoda, K. (2023) · Journal of Epidemiology · doi:10.2188/jea.je20230078 · Cited by 38
ELECTRAMed: a new pre-trained language representation model for biomedical NLPPreprint
Miolo, G., Mantoan, G., & Orsenigo, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09585 · Cited by 35
TALM: Tool Augmented Language Models
Parisi, A., Zhao, Y., & Fiedel, N. (2022) · arXiv · doi:10.48550/arxiv.2205.12255 · Cited by 35
Siegerink, B., Pet, L. A., Rosendaal, F. R., & Schoones, J. W. (2023) · Nurse Education in Practice · doi:10.1016/j.nepr.2023.103599 · Cited by 31
Overview of the Medical Question Answering Task at TREC 2017 LiveQA.
Abacha, A. B., Agichtein, E., Pinter, Y., & Demner‐Fushman, D. (2017) · Text REtrieval Conference · Cited by 30
Automated Metrics for Medical Multi-Document Summarization Disagree with Human Evaluations
Wang, L. L., Otmakhova, Y., DeYoung, J., Truong, T. H., Kuehl, B., Bransom, E., & Wallace, B. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.549 · Cited by 22
Can ChatGPT Be Considered an Author of a Medical Article?
Ide, K., Hawke, P., & Nakayama, T. (2023) · Journal of Epidemiology · doi:10.2188/jea.je20230030 · Cited by 21
Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges
Ramprasad, S., Mcinerney, J., Marshall, I. J., & Wallace, B. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-demo.27 · Cited by 12
Query-Focused EHR Summarization to Aid Imaging DiagnosisPreprint
McInerney, D. J., Dabiri, B. E., Touret, A.-S., Young, G. S., van de Meent, J.-W., & Wallace, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.04645 · Cited by 10
Improving Radiology Summarization with Radiograph and Anatomy Prompts
Hu, J., Chen, Z., Liu, Y., Wan, X., & Chang, T. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.764 · Cited by 10
Large Language Models Like ChatGPT in ABME: Author Guidelines
Norris, C. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03212-2 · Cited by 8
Large Language Models in Medical Education and Quality Concerns
Mishra, V. (2023) · Journal of Quality in Health Care & Economics · doi:10.23880/jqhe-16000319 · Cited by 2
Research works printed but not linked: 32
- OpenAI, 2022, Introducing ChatGPT
- OpenAI, 2023, GPT-4 Technical Report
- Bolton, 2022, BioMedLM: a Domain-Specific Large Language Model for Biomedical Text
- Wu, 2023, PMC-LLaMA: Further Finetuning LLaMA on Medical Papers
- Radford, 2018, Improving Language Understanding by Generative Pre-Training
- Radford, 2019, OpenAI Research Papers
- Chowdhery, 2022, et al
- Taylor, 2022, et al
- Touvron, 2023, et al
- Toma, 2023, Clinical Camel: An Open-Source Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding
- Li, 2023, ChatDoctor: A Medical Chat Model Fine-tuned on LLaMA Model using Medical Domain Knowledge
- Anil R, Dai AM, Firat O, 2023, PaLM 2 Technical Report
- Abacha, 2019, MEDINFO 2019: Health and Wellbeing e-Networks for All, 25
- Kanoulas, 2018, CEUR Workshop Proceedings
- Kanoulas, 2019, CEUR Workshop Proceedings
- Hou, 2023, GeneTuring tests GPT models in genomics, 10.1101/2023.03.11.532238 · DOI
- Fox, 2013, Pew Research Center Blog Post
- Cohan, 2018, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Laundry, 2023, Turn GPT-4 Into your Personal Literature Review Bot. The Academic’s Field Guide to Writing Code Blog Post on Medium
- Smit, 2020, et al
- Wang, 2023, et al
- Tang, 2023, npj Digit Med
- Luo, Bioinformatics
- Krallinger, 2017, Proceedings of the sixth BioCreative Challenge Evaluation Workshop
- Chen, 2023, Large Language Models in Biomedical Natural Language Processing: Benchmarks, Baselines, and Recommendations
- OpenAI, 2023, Educator considerations for ChatGPT
- Gasperin, 2007, Proceedings of DAARC
- Su, Dagstuhl Seminar Proceedings
- Batista-Navarro, Proceedings of BioNLP 2011 Workshop
- OpenAI, 2023, GPT-4 System Card
- Papineni, Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics
- Lin, Text Summarization Branches Out
Cited by
Sorted by citations
Large language models in healthcare and medical domain: A review
Nazi, Z. A., & Peng, W. (2024) · Informatics · doi:10.3390/informatics11030057 · Cited by 415
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358
Improving large language models for clinical named entity recognition via prompt engineering
Hu, Y., Chen, Q., Du, J., Peng, X., Keloth, V. K., Zuo, X., Zhou, Y., Li, Z., Jiang, X., Lu, Z., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad259 · Cited by 314
Self-citationA systematic review of large language models and their implications in medical education
Lucas, H. C., Upperman, J. S., & Robinson, J. R. (2024) · Medical Education · doi:10.1111/medu.15402 · Cited by 287
Matching patients to clinical trials with large language models
Jin, Q., Wang, Z., Floudas, C. S., Chen, F., Gong, C., Bracken-Clarke, D., Xue, E., Yang, Y., Sun, J., & Lu, Z. (2024) · Nature Communications · doi:10.1038/s41467-024-53081-z · Cited by 287
Self-citation
Metadata
- Authors
- Tian, S., Jin, Q., Yeganova, L., Lai, P.-T., Zhu, Q., Chen, X., Yang, Y., Chen, Q., Kim, W., Comeau, D. C., Islamaj, R., Kapoor, A., Gao, X., & Lu, Z.
- Affiliations
- United States National Library of Medicine (US)
- King Abdullah University of Science and Technology (SA)
- Corresponding author
- Lu, Zhiyong
- ORCID
- Venue
- Briefings in Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-11-22
- Volume
- 25
- Issue
- 1
- ISSN
- 1467-5463 (ISSN-L)
- 1477-4054
- PMC
- PMC10762511
- OpenAlex
- W4381587418
- Cited by
- 359
- References
- 132
- Citations by year
- 2026: 81 · 2025: 161 · 2024: 112 · 2023: 5
- FWCI
- 14.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- public-domain
- Locations
- Briefings in Bioinformatics (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- PubMed Central (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- NIH Intramural Research Program
- National Library of Medicine
- National Institutes of Health (1K99LM014024)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- ChatGPT, large language models, biomedical information retrieval, question answering, medical text summarization, information extraction, medical education, patient data privacy, legal issues, text generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
