What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams
Open accessPublished version of a preprint
The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Open domain question answering (OpenQA) tasks have been recently attracting more and more attention from the natural language processing (NLP) community. In this work, we present the first free-form multiple-choice OpenQA dataset for solving medical problems, MedQA, collected from the professional medical board exams. It covers three languages: English, simplified Chinese, and traditional Chinese, and contains 12,723, 34,251, and 14,123 questions for the three languages, respectively. We implement both rule-based and popular neural methods by sequentially combining a document retriever and a machine comprehension model. Through experiments, we find that even the current best method can only achieve 36.7%, 42.0%, and 70.1% of test accuracy on the English, traditional Chinese, and simplified Chinese questions, respectively. We expect MedQA to present great challenges to existing OpenQA systems and hope that it can serve as a platform to promote much stronger OpenQA models from the NLP community in the future.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“given that medical question datasets such as PubMedQA42 and MedQA43 consist of fixed question sets susceptible to memorization”
From An automated framework for assessing how well LLMs cite relevant medical references (2025) · cites this paper as [42]
In the citing paper
“Notably, in healthcare, medical NLP benchmarks like MedQA7, PubMedQA8 and the Massive Multitask Language Understanding (MMLU) serve as the de-facto reporting standard for state-of-the-art medical LLMs9–11 with claims of ‘superhuman’ performance in patient-facing tasks12.”
From Medical large language models are vulnerable to data-poisoning attacks (2025) · cites this paper as [7]
In the citing paper
“Current practice relies heavily on static, fixed-length benchmarks such as, MedQA2, MMLU-Med3, and MedBench4.”
From Leveraging computerized adaptive testing for cost-effective evaluation of large language models in medical benchmarking (2026) · cites this paper as [2]
In the citing paper
“Large-scale medical benchmarks often comprise tens of thousands of multiple-choice questions (MCQs) (e.g., MedQA: 12,723 items2; MedMCQA: 6,150 items9; MedBench: 40,041 items4; Med-HALT: 59,254 items10), leading to substantial token consumption and API costs—particularly for proprietary models.”
From Leveraging computerized adaptive testing for cost-effective evaluation of large language models in medical benchmarking (2026) · cites this paper as [2]
In the citing paper
“(5) MedQA, introduced by Jin et al. (2021), using the USMLE subset.”
From Sabotage Evaluations for Frontier Models (2024) · cites this paper as Jin et al. (2021)
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
Know What You Don't Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · arXiv · doi:10.48550/arxiv.1806.03822 · Cited by 2,242
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.740 · Cited by 2,103
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834
Research works printed but not linked: 3
- Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv.
- Dhingra, B., Mazaitis, K., and Cohen, W.W. (2017). Quasar: Datasets for Question Answering by Search and Reading. arXiv.
- Das, R., Dhuliawala, S., Zaheer, M., and McCallum, A. (2019). Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering. arXiv.
Cited by
Items 201–300 of 762
A survey on large language models in biology and chemistry
Ashyrmamatov, I., Gwak, S. J., Jin, S.-Y., Jun, I., Ucak, U. V., Lee, J. Y., & Lee, J. (2025) · Experimental & Molecular Medicine · doi:10.1038/s12276-025-01583-1 · Cited by 17
Salehin, I., Sajib, M. T. A., Badhon, N. H., Rifat, M. S. H., Amin, N., & Moon, N. N. (2025) · Engineering Reports · doi:10.1002/eng2.70365 · Cited by 13
Zhao, W., Deng, Z., Yadav, S., & Yu, P. S. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651941 · Cited by 18
LongHealth: A Question Answering Benchmark with Long Clinical Documents
Adams, L. C., Busch, F., Han, T., Excoffier, J.-B., Ortala, M., Löser, A., Aerts, H. J. W. L., Kather, J. N., Truhn, D., & Bressem, K. K. (2025) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-025-00204-w · Cited by 70
Ahn, J., Kang, B. G., Chang, M. Y., & Yoon, S. (2025) · Clinical and Experimental Otorhinolaryngology · doi:10.21053/ceo.2025-00121 · Cited by 10
Advancing medical question answering with a knowledge embedding transformer
Zhu, X., Khan, M., Ahmed, A. T., & Othmani, A. (2025) · PLoS ONE · doi:10.1371/journal.pone.0329606 · Cited by 12
Li, R., Zhao, A., Lei, P., Shi, H., Zhao, J., Li, Z., Liang, R., & Wang, H. (2025) · Annals of Surgical Oncology · doi:10.1245/s10434-025-18492-2 · Cited by 12
Cross-document attention-based gated fusion network for automated medical licensing exam
Liu, J., Ren, J., Lu, Z., He, W., Cui, M., Zhang, Z., & Bai, R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117588 · Cited by 10
MED-ChatGPT CoPilot: a ChatGPT medical assistant for case mining and adjunctive therapy
Liu, W., Kan, H., Jiang, Y., Geng, Y., Nie, Y., & Yang, M. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1460553 · Cited by 11
Nazar, W., Nazar, G., Kamińska, A., & Daniłowicz‐Szymanowicz, L. (2025) · Journal of Medical Internet Research · doi:10.2196/70481 · Cited by 20
PNCD: Mitigating LLM hallucinations in noisy environments–A medical case study
Qu, J., Liu, J., Liu, X., Chen, M., Li, J., & Wang, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103328 · Cited by 13
Verification is All You Need: Prompting Large Language Models for Zero-Shot Clinical Coding
Li, S., Zheng, C., Wu, J., Xu, Q., Xu, X., Wang, H., Sun, Y., Bai, Z., Xu, Y., Zhu, L., et al. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3593028 · Cited by 13
Reviewing clinical knowledge in medical large language models: Training and beyond
Li, Q. Y., Liu, H., Guo, C., Gao, C., Chen, D., Wang, M., Gao, F., van Harmelen, F., & Gu, J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114215 · Cited by 20
MedAgentSim: Self-evolving Multi-agent Simulations for Realistic Clinical Interactions
Almansoori, M., Kumar, K., & Cholakkal, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05114-1_35 · Cited by 42
Large Language Models for Computer-Aided Design: A Survey
Zhang, L., Le, B., Akhtar, N., Lam, S.-K., & Ngo, D. (2026) · ACM Computing Surveys · doi:10.1145/3787499 · Cited by 74
Large language models for clinical artificial intelligence in healthcare a systematic review
Ghnemat, R., & Saleh, A. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00784-x · Cited by 17
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models
Youssef, P., Koraş, O. A., Li, M., Schlötterer, J., & Seifert, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1043 · Cited by 52
Biomedical Question Answering: A Comprehensive Review.Preprint
Jin, Q., Yuan, Z. P., Xiong, G., Yu, Q., Tan, C., Chen, M., Huang, S., Liu, X., & Yu, S. (2021) · arXiv (Cornell University) · Cited by 11
tinyBenchmarks: evaluating LLMs with fewer examples
Polo, F. M., Weber, L., Choshen, L., Sun, Y., Xu, G., & Yurochkin, M. (2024) · arXiv · doi:10.48550/arxiv.2402.14992 · Cited by 8
Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models
Luo, L., Zhao, Z., Haffari, G., Li, Y.-F., Gong, C., & Pan, S. (2024) · arXiv · doi:10.48550/arxiv.2410.13080 · Cited by 8
Wang, A., Wu, Y., Ji, X., Wang, X., Hu, J., Zhang, F., Zhang, Z., Pu, D., Tang, L., Ma, S., et al. (2024) · JMIR Research Protocols · doi:10.2196/57001 · Cited by 10
de Matos, J. C. R. G., Gallifant, J., Pei, J., & Wong, A.-K. I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82007-6_20 · Cited by 13
An Empirical Evaluation of Large Language Models on Consumer Health Questions
Abrar, M., Sermet, Y., & Demir, İ. (2025) · BioMedInformatics · doi:10.3390/biomedinformatics5010012 · Cited by 16
Wu, E., Wu, K., & Zou, J. (2025) · NEJM AI · doi:10.1056/aics2401155 · Cited by 12
Liu, B., Nie, Y., Song, H., Lin, Y., Li, J., Weng, X., Su, Z., Suo, Y., Lv, T., Zhao, X., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103557 · Cited by 12
MedCoT-RAG: Causal Chain-of-Thought RAG for Medical Question Answering
Wang, Z., Khatibi, E., & Rahmani, A. M. (2025) · Wearable and Implantable Body Sensor Networks · doi:10.1109/bsn66969.2025.11337389 · Cited by 23
A clinical environment simulator for dynamic AI evaluation
Luo, L., Kim, S. E., Zhang, X., Kernbach, J. M., Kenia, R., Acosta, J. N., Nathanson, L. A., Haimovich, A. D., Rodman, A. M., Goh, E., et al. (2026) · Nature Medicine · doi:10.1038/s41591-026-04252-6 · Cited by 36
Relation-Aware Language-Graph Transformer for Question Answering
Park, J., Choi, H. K., Ko, J., Park, H., Kim, J., Jeong, J., Kim, K., & Kim, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26578 · Cited by 22
Multi-Task Training with In-Domain Language Models for Diagnostic Reasoning
Sharma, B., Gao, Y., Miller, T. A., Churpek, M. M., Afshar, M., & Dligach, D. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.10 · Cited by 13
An Empirical Evaluation of Large Language Models on Consumer Health Questions
Abrar, M., Sermet, Y., & Demir, I. (2025) · arXiv · doi:10.48550/arxiv.2501.00208 · Cited by 8
Infusing Multi-Hop Medical Knowledge Into Smaller Language Models for Biomedical Question Answering
Chen, J., Wei, Z., Shen, W., & Shang, R. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3547444 · Cited by 10
Enhancing medical digital twins within metaverse using blockchain, NFTs and LLMs
Islayem, R., Musamih, A., Salah, K. M., Jayaraman, R., & Yaqoob, I. (2025) · Internet of Things · doi:10.1016/j.iot.2025.101648 · Cited by 13
A survey for wearable sensors empowering smart healthcare in the era of large language models
Liu, Y., Li, J., Lian, C., Zhang, X., Gong, J., Wang, Y., & Zhao, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103409 · Cited by 11
Benchmarking large language models for replication of guideline-based PGx recommendations
Zastrozhin, M. S., Slobodchikov, I. D., Stupichev, D., Moore, A. K., Sokolov, D., Trifonov, I., & Gobbs, A. (2025) · The Pharmacogenomics Journal · doi:10.1038/s41397-025-00383-0 · Cited by 8
Medication information extraction using local large language models
Richter-Pechanski, P., Seiferling, M., Kiriakou, C., Schwab, D. M., Geis, N. A., Dieterich, C., & Frank, A. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104898 · Cited by 16
Liu, Z., Kou, J., Zhang, W., Gu, C., Fang, X., Huang, Z., Hao, Y., Li, H., Lu, X., Yin, A., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189137 · Cited by 12
Dylag, J., Zlatev, Z., & Boniface, M. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03055-y · Cited by 9
He, J., Zhang, B., Rouhizadeh, H., Chen, Y., Yang, R., Lu, J. S., Chen, X., Liu, N., & Teodoro, D. (2026) · Artificial Intelligence Review · doi:10.21203/rs.3.rs-8330917/v1 · Cited by 33
Large language models in healthcare and biomedical informatics: A comprehensive review
Hornback, A. M., Sathu, H., Kim, K., Wang, Y., Zhu, Y., Isgut, M., Avula, P., Khimani, A., & Wang, M. D. (2026) · Innovation and Emerging Technologies · doi:10.1142/s2737599426300011 · Cited by 9
BRIDGE: benchmarking large language models for understanding real-world clinical practice texts
Wu, J., Gu, B., Zhou, R., Xie, K., Snyder, D., Jiang, Y., Carducci, V., Wyss, R., Desai, R. J., Alsentzer, E., et al. (2026) · Nature Biomedical Engineering · doi:10.1038/s41551-026-01719-2 · Cited by 50
Explanatory argument extraction of correct answers in resident medical exams
Goenaga, I., Atutxa, A., Gojenola, K., Oronoz, M., & Agerri, R. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102985 · Cited by 17
Survey of Large Language Models for Answering Questions Across Various Fields
Shailendra, P., Ghosh, R. C., Kumar, R., & Sharma, N. (2024) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs60874.2024.10717078 · Cited by 24
Socrates, V., Gilson, A., Lopez, K., Chi, L., Taylor, R. A., & Chartash, D. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104360 · Cited by 12
Şensoy, E., & Çıtırık, M. (2024) · Contact Lens and Anterior Eye · doi:10.1016/j.clae.2024.102125 · Cited by 9
Igarashi, Y., Nakahara, K., Norii, T., Miyake, N., Tagami, T., & Yokobori, S. (2024) · Journal of Nippon Medical School · doi:10.1272/jnms.jnms.2024_91-205 · Cited by 11
Azizoğlu, M., & Okur, M. H. (2023) · Research Square · doi:10.21203/rs.3.rs-3018641/v1 · Cited by 8
AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator
Fan, Z., Tang, J., Chen, W., Wang, S., Wei, Z., Xi, J., Huang, F., & Zhou, J. (2024) · arXiv · doi:10.48550/arxiv.2402.09742 · Cited by 84
Domain-Tailored Generative AI for Personalized Assistant
Jiang, N., Hasanzadeh, S., & Duffy, V. G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-76809-5_17 · Cited by 7
Enhancing healthcare resource allocation through large language models
Wan, F. J., Wang, K., Wang, T., Qin, H., Fondrevelle, J., & Duclos, A. (2025) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2025.101859 · Cited by 11
A Self-Evolving Framework for Multi-Agent Medical Consultation Based on Large Language Models
Chen, K., Qi, J., Huo, J., Tian, P., Meng, F., Yang, X., & Gao, Y. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10889517 · Cited by 6
LENS: Layers of Evaluation of Hallucination in GenAI Systems
Gu, C., Zhang, W., Huang, Z., Kou, J., Liu, Z., Zhao, C., Liu, C., Zhang, L., Lin, W., Wang, Z., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189150 · Cited by 36
May, P., Nokodian, S., Nuernbergk, C., Knauer, M., Hefter, M., von Rose, A. B., Bassermann, F., & Jung, J. (2026) · JCO Clinical Cancer Informatics · doi:10.1200/cci-25-00194 · Cited by 6
Security and Privacy of Large Language Models: Threat Taxonomy, Ethical Implications, and Governance
Pribisalić, M., & Martinčić-Ipšić, S. (2026) · AI · doi:10.3390/ai7050152 · Cited by 9
Neurological Diagnosis: Artificial Intelligence Compared With Diagnostic Generator
Finelli, P. F. (2024) · The Neurologist · doi:10.1097/nrl.0000000000000560 · Cited by 6
Large Language Models for Healthcare: Applications, Models, Datasets, and Challenges
Rezgui, K. (2024) · International Conference on Control, Decision and Information Technologies (CoDIT) · doi:10.1109/codit62066.2024.10708253 · Cited by 9
Meta-learning for compressed language model: A multiple choice question answering study
Yan, M., & Pan, Y. F. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.01.148 · Cited by 7
Research on a massively large artificial intelligence model and its application in medicine
Guo, H., Liu, P., Lu, R., Yang, F., Xu, H., Zhuang, Y., Huang, G. M., Song, S., & He, K. (2023) · Scientia Sinica Vitae · doi:10.1360/ssv-2022-0298 · Cited by 10
Prompt Discriminative Language Models for Domain Adaptation
Lu, K., Potash, P., Lin, X., Sun, Y., Qian, Z., Zheng, Y. G., Naumann, T., Cai, T., & Lu, J. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.30 · Cited by 11
Kanakarajan, K. R., & Sankarasubbu, M. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.137 · Cited by 23
Mixture-of-Experts for Biomedical Question Answering
Dai, D., Jiang, W., Zhang, J., Lyu, Y., Sui, Z., & Chang, B. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44693-1_47 · Cited by 15
ChatGPT's learning and reasoning capacity in anesthesiology
Cruz, G., Pedroza, S., & Ariza, F. (2023) · Colombian Journal of Anesthesiology · doi:10.5554/22562087.e1092 · Cited by 8
Designing Retrieval-Augmented Language Models for Clinical Decision Support
Quigley, K., Koker, T., Taylor, J., Mancuso, V., & Brattain, L. J. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-63592-2_13 · Cited by 8
Şensoy, E., & Çıtırık, M. (2024) · Journal of Current Ophthalmology · doi:10.4103/joco.joco_192_23 · Cited by 7
Complex QA and language models hybrid architectures, SurveyPreprint
Daull, X., Bellot, P., Bruno, E., Martin, V., & Murisasco, É. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09051 · Cited by 43
Improved precision oncology question-answering using agentic LLMPreprint
Das, R., Maheswari, K., Siddiqui, B. S., Arora, N., Paul, A., Nanshi, J., Udbalkar, V., Sarvade, A., Chaturvedi, H., Shvartsman, T., et al. (2024) · medRxiv · doi:10.1101/2024.09.20.24314076 · Cited by 13
Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
Smit, A., Duckworth, P., Grinsztajn, N., Barrett, T. D., & Pretorius, A. (2023) · arXiv · doi:10.48550/arxiv.2311.17371 · Cited by 73
Yao, M. S., Chae, A., Saraiya, P. V., Kahn, C. E., Witschey, W. R. T., Gee, J. C., Sagreiya, H., & Bastani, O. (2025) · Communications Medicine · doi:10.1038/s43856-025-01061-9 · Cited by 10
C-PATH: Conversational Patient Assistance and Triage in Healthcare System
Shi, Q. F., Han, Q., & Soares, C. (2025) · IEEE International Conference on Digital Health (ICDH) · doi:10.1109/icdh67620.2025.00016 · Cited by 7
Prompt design for medical question answering with Large Language Models
Kuligin, L., Lammert, J., Ostapenko, A., Bressem, K. K., Boeker, M., & Tschochohei, M. (2025) · Machine Learning with Applications · doi:10.1016/j.mlwa.2025.100758 · Cited by 12
Enhancing Clinical Decision Support with LLMs: A Feasibility Study Integrating CoT, RAG, and QLoRA
Matthew, G., & Hicks, Y. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.09.621 · Cited by 5
Susceptibility of Large Language Models to User-Driven Factors in Medical Queries
Lim, K. H., Kang, U., Li, X., Kim, J. S., Jung, Y., Park, S. J., & Kim, B.-H. (2025) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-025-00218-4 · Cited by 17
A multi-agent approach to neurological clinical reasoning
Sorka, M., Gorenshtein, A., Aran, D., & Shelly, S. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0001106 · Cited by 11
Advancing Healthcare with Large Language Models: Techniques and Application
Hu, Z., Peng, Z., Bi, Z., Shen, Q., Liu, Z., Lou, J., & Luo, X. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125540 · Cited by 12
MedTrust-RAG: Evidence Verification and Trust Alignment for Biomedical Question Answering
Ning, Y., Sun, Y.-Y., Luo, L., Wang, Y.-H., Pan, Y., & Lin, H.-F. (2025) · IEEE International Conference on Bioinformatics and Biomedicine · doi:10.1109/bibm66473.2025.11356290 · Cited by 6
SearchRAG: Can Search Engines Be Helpful for LLM-Based Medical Question Answering?
Shi, Y., Yang, T., Chen, C., Li, Q., Liu, T., Li, X., & Liu, N. K. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356025 · Cited by 21
Shuriaqi, S. A., Alzaabi, A. A., & Abdulsalam, A. A. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8020041 · Cited by 6
Reason-Align-Respond: Aligning LLM Reasoning With Knowledge Graphs for KGQA
Shen, X., Wang, F., Yang, Z., Wang, B., Du, W., Zong, C., & Xia, R. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3665645 · Cited by 15
A systematic review of the limitations of large language models in generating healthcare content
Khosravi, M., Zamaninasab, Z., Mojtabaeian, S. M., Demiray, E. K. D., & Arab‐Zozani, M. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001354 · Cited by 12
Medical Reasoning With Large Language Models: A Systematic Review and Evaluation
Ren, X., Fan, C., Ma, W., He, H., Gao, C., Zhao, X., & Feng, F. (2026) · iNew Medicine · doi:10.1002/inm3.70056 · Cited by 9
Şensoy, E., & Çıtırık, M. (2023) · Middle East African Journal of Ophthalmology · doi:10.4103/meajo.meajo_199_23 · Cited by 5
García-Agúndez, A., Schmajuk, G., & Yazdany, J. (2024) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2024.152584 · Cited by 6
Optimizing Communication and Performance in Federated Learning for Large Language Models
Song, I., & Lee, K. Y. (2025) · International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic64266.2025.10920742 · Cited by 6
Mondillo, G., Colosimo, S., Perrotta, A., Frattolillo, V., Masino, M., Martino, M., del Giudice, E. M., & Marzuillo, P. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106027 · Cited by 7
Avnat, E., Levy, M., Herstain, D., Yanko, E., Joya, D. B., Katz, M. T., Eshel, D., Laros, S., Dagan, Y., Barami, S., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/64452 · Cited by 13
Liu, J., & Liu, S. (2025) · Journal of Medical Systems · doi:10.1007/s10916-025-02232-w · Cited by 7
A Comparative Review of GPT-4’s Applications in Medicine and High Decision Making
Bitri, R., & Ali, M. (2023) · International Conference on Computing, Networking, Telecommunications & Engineering Sciences Applications (CoNTESA) · doi:10.1109/contesa61248.2023.10384948 · Cited by 6
Information bottleneck based knowledge selection for commonsense reasoning
Yang, Z., Zhang, Y., Cao, P., Cao, L., Chen, J., Zhao, J., & Liu, K. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120134 · Cited by 7
GongBu: Easily Fine-tuning LLMs for Domain-specific Adaptation
Zhang, B., Tian, Y., Wang, S., Tu, Z., Chu, D., & Shen, Z. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679233 · Cited by 5
PedMedQA: Comparing Large Language Model Accuracy in Pediatric and Adult Medicine
Jaiswal, N., Ma, Y., Lebouché, B., Poenaru, D., & Osmanlliu, E. (2025) · Pediatrics Open Science · doi:10.1542/pedsos.2025-000485 · Cited by 4
Leveraging Datathons to Teach AI in Undergraduate Medical Education: Case Study
Yao, M. S., Huang, L., Leventhal, E. L., Sun, C., Stephen, S. J., & Liou, L. (2025) · JMIR Medical Education · doi:10.2196/63602 · Cited by 6
Guo, A., Canagasingham, A., Rasiah, K., Chalasani, V., Mundy, J. A., & Chung, A. S. J. (2025) · ANZ Journal of Surgery · doi:10.1111/ans.70186 · Cited by 5
Gül, M. C. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07856-7 · Cited by 4
Retrieval augmented large language model system for comprehensive drug contraindications
Bang, B., Yoon, J. C., Chang, D. J., Park, S., & Lee, Y. O. (2026) · Health Information Science and Systems · doi:10.1007/s13755-025-00420-z · Cited by 7
Şahin, M. Z., Derdiyok, F. B., Kılıç, S. A., Serbest, K., & Nas, K. (2026) · Bioengineering · doi:10.3390/bioengineering13010125 · Cited by 5
An Information–Theoretic Model of Abduction for Detecting Hallucinations in Explanations
Galitsky, B. A., & Rybalov, A. (2026) · Entropy · doi:10.3390/e28020173 · Cited by 7
Wang, J., Ning, H., Peng, Y., Wei, Q., Tesfai, D., Mao, W., Zhu, T., & Huang, R. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.132914 · Cited by 23
Sachpelidis-Brozos, N., Katsaros, E., Grammatikis, P. I. R., Kalitsios, G., Sarigiannidis, A. G., Seritan, G. C., Politis, I., Xenakis, C., Goudos, S. K., & Sarigiannidis, P. G. (2026) · Computer Science Review · doi:10.1016/j.cosrev.2026.100923 · Cited by 8
Rogoz, A.-C., Ionescu, R. T., Anghel, A.-V., Antone-Iordache, I.-L., Coniac, S., & Ionescu, A.-I. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02465-0 · Cited by 10
Reducing Hallucinations in Medical AI Through Citation Enforced Prompting in RAG Systems
Pawlik, Ł., & Deniziak, S. R. (2026) · Applied Sciences · doi:10.3390/app16063013 · Cited by 6
Chen, P., Cai, J., Zhou, J., Chen, S., Xu, C., Yuan, L., Dai, X., Chen, X., Wei, Y., Li, X., et al. (2026) · Communications Medicine · doi:10.1038/s43856-026-01576-9 · Cited by 9
Metadata
- Authors
- Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P.
- Affiliations
- Massachusetts Institute of Technology (US)
- Huazhong University of Science and Technology (CN)
- Computer Science and Artificial Intelligence, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
- Corresponding author
- Jin, Di
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2021-07-12
- Volume
- 11
- Issue
- 14
- Pages
- 6421
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3162922479
- Cited by
- 718
- References
- 41
- Citations by year
- 2026: 285 · 2025: 241 · 2024: 120 · 2023: 51 · 2022: 12 · 2021: 9
- FWCI
- 16.96
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- Preprints.org (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- open-domain question answering, medical question answering, multiple-choice questions, Chinese language, document retrieval, machine reading comprehension, disease diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.