What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams
Open accessPublished version of a preprint
The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Open domain question answering (OpenQA) tasks have been recently attracting more and more attention from the natural language processing (NLP) community. In this work, we present the first free-form multiple-choice OpenQA dataset for solving medical problems, MedQA, collected from the professional medical board exams. It covers three languages: English, simplified Chinese, and traditional Chinese, and contains 12,723, 34,251, and 14,123 questions for the three languages, respectively. We implement both rule-based and popular neural methods by sequentially combining a document retriever and a machine comprehension model. Through experiments, we find that even the current best method can only achieve 36.7%, 42.0%, and 70.1% of test accuracy on the English, traditional Chinese, and simplified Chinese questions, respectively. We expect MedQA to present great challenges to existing OpenQA systems and hope that it can serve as a platform to promote much stronger OpenQA models from the NLP community in the future.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“given that medical question datasets such as PubMedQA42 and MedQA43 consist of fixed question sets susceptible to memorization”
From An automated framework for assessing how well LLMs cite relevant medical references (2025) · cites this paper as [42]
In the citing paper
“Notably, in healthcare, medical NLP benchmarks like MedQA7, PubMedQA8 and the Massive Multitask Language Understanding (MMLU) serve as the de-facto reporting standard for state-of-the-art medical LLMs9–11 with claims of ‘superhuman’ performance in patient-facing tasks12.”
From Medical large language models are vulnerable to data-poisoning attacks (2025) · cites this paper as [7]
In the citing paper
“Current practice relies heavily on static, fixed-length benchmarks such as, MedQA2, MMLU-Med3, and MedBench4.”
From Leveraging computerized adaptive testing for cost-effective evaluation of large language models in medical benchmarking (2026) · cites this paper as [2]
In the citing paper
“Large-scale medical benchmarks often comprise tens of thousands of multiple-choice questions (MCQs) (e.g., MedQA: 12,723 items2; MedMCQA: 6,150 items9; MedBench: 40,041 items4; Med-HALT: 59,254 items10), leading to substantial token consumption and API costs—particularly for proprietary models.”
From Leveraging computerized adaptive testing for cost-effective evaluation of large language models in medical benchmarking (2026) · cites this paper as [2]
In the citing paper
“(5) MedQA, introduced by Jin et al. (2021), using the USMLE subset.”
From Sabotage Evaluations for Frontier Models (2024) · cites this paper as Jin et al. (2021)
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
Know What You Don't Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · arXiv · doi:10.48550/arxiv.1806.03822 · Cited by 2,242
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.740 · Cited by 2,103
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834
Research works printed but not linked: 3
- Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv.
- Dhingra, B., Mazaitis, K., and Cohen, W.W. (2017). Quasar: Datasets for Question Answering by Search and Reading. arXiv.
- Das, R., Dhuliawala, S., Zaheer, M., and McCallum, A. (2019). Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering. arXiv.
Cited by
Items 101–200 of 762
ClinicalAgent: Clinical Trial Multi-Agent System with Large Language Model-based Reasoning
Yue, L., Xing, S., Chen, J., & Fu, T. (2024) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3698587.3701359 · Cited by 113
Pashangpour, S., & Nejat, G. (2024) · Robotics · doi:10.3390/robotics13080112 · Cited by 50
Evaluating Large Language Models in Ophthalmology: Systematic Review
Zhang, Z., Zhang, H., Pan, Z., Bi, Z., Wan, Y., Song, X., & Fan, X. (2025) · Journal of Medical Internet Research · doi:10.2196/76947 · Cited by 48
(2025) · doi:10.1007/s11432-025-4665-8 · Cited by 34
McLean, A. L., Wu, Y., McLean, A. C. L., & Hristidis, V. (2024) · Journal of Cancer Research and Clinical Oncology · doi:10.1007/s00432-024-05673-x · Cited by 39
Medical Hallucinations in Foundation Models and Their Impact on Healthcare
Kim, Y., Jeong, H., Chen, S., Li, S. S., Park, C., Lu, M., Alhamoud, K., Mun, J., Grau, C., Jung, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.05777 · Cited by 33
Xiao, L., Wang, D., Zhong, H., Wang, Q., Li, R., Jia, R., & Wan, B. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103805 · Cited by 39
VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge
Nath, V., Li, W., Yang, D., Myronenko, A., Zheng, M., Lu, Y., Liu, Z., Yin, H., Law, Y. M., Tang, Y., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01378 · Cited by 119
Jeon, S., & Kim, H. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110614 · Cited by 55
From System 1 to System 2: A Survey of Reasoning Large Language Models
Zhang, D., Li, Z., Zhang, M.-L., Zhang, J., Liu, Z., Yao, Y., Xu, H., Zheng, J., Chen, X., Zhang, Y., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3637037 · Cited by 612
Carrasco, J. P., García, E., Martínez, D. A. S., Estrella-Porter, P., la Puente, L. D., Navarro, J., & Cerame, Á. (2023) · Revista Española de Educación Médica · doi:10.6018/edumed.556511 · Cited by 62
Yang, X., Li, T., Wang, H., Zhang, R., Ni, Z., Liu, N., Zhai, H., Zhao, J., Meng, F., Zhou, Z., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01486-5 · Cited by 35
Kformer: Knowledge Injection in Transformer Feed-Forward Layers
Yao, Y., Huang, S., Li, D., Wei, F., Chen, H., & Zhang, N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17120-8_11 · Cited by 59
Generative artificial intelligence: implications for biomedical and health professions education
Hersh, W. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-094756 · Cited by 28
HealthQ: Unveiling questioning capabilities of LLM chains in healthcare conversations
Wang, Z., Li, H., Huang, D., Kim, H., Shin, C.-W., & Rahmani, A. M. (2025) · Smart Health · doi:10.1016/j.smhl.2025.100570 · Cited by 71
BioReader: a Retrieval-Enhanced Text-to-Text Transformer for Biomedical Literature
Frisoni, G., Mizutani, M., Moro, G., & Valgimigli, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.390 · Cited by 21
Automating expert-level medical reasoning evaluation of large language models
Zhou, S., Xie, W., Li, J., Zhan, Z., Song, M., Yang, H., Espinoza, C., Welton, L., Mai, X., Jin, Y., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02208-7 · Cited by 30
Flaharty, K. A., Hu, P., Hanchard, S. L., Ripper, M. E., Duong, D., Waikel, R. L., & Solomon, B. D. (2024) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2024.07.011 · Cited by 26
The effects of generative AI’s human-like competencies on clinical decision-making
Spatscheck, N., Schaschek, M., & Winkelmann, A. (2024) · Journal of Decision System · doi:10.1080/12460125.2024.2430731 · Cited by 36
Gu, Z., Jia, W., Piccardi, M., & Yu, P. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103078 · Cited by 32
Large Language Models in Psychiatry: Current Applications, Limitations, and Future Scope
Liu, Z., Bao, Y., Zeng, S., Qian, R., Deng, M., Gu, A., Li, J., Wang, W., Cai, W., Li, W., et al. (2024) · Big Data Mining and Analytics · doi:10.26599/bdma.2024.9020046 · Cited by 31
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT
Meng, Z., Liu, F., Clark, T. H., Shareghi, E., & Collier, N. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.383 · Cited by 38
MLEC-QA: A Chinese Multi-Choice Biomedical Question Answering Dataset
Li, J., Zhong, S., & Chen, K. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.698 · Cited by 72
DR.BENCH: Diagnostic Reasoning Benchmark for Clinical Natural Language Processing
Gao, Y., Dligach, D., Miller, T. A., Caskey, J., Sharma, B., Churpek, M. M., & Afshar, M. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104286 · Cited by 54
InferDPT: Privacy-Preserving Inference for Closed-Box Large Language Models
Tong, M., Chen, K., Zhang, J., Qi, Y., Zhang, W., Yu, N., Zhang, T., & Zhang, Z. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3550389 · Cited by 78
Development and evaluation of a large language model of ophthalmology in Chinese
Zheng, C., Ye, H., Guo, J., Yang, J., Fei, P., Yuan, Y., Huang, D., Huang, Y., Peng, J., Xie, X., et al. (2024) · British Journal of Ophthalmology · doi:10.1136/bjo-2023-324526 · Cited by 29
Haoyu, T., Kuo, Y., Xin, D., Chenxi, Z., Mingwei, Y., Hongyan, W., Yiming, L., Minjie, H., Qiang, Z., Yu, J., et al. (2024) · Digital Chinese Medicine · doi:10.1016/j.dcmed.2025.01.007 · Cited by 29
Exploring the effectiveness of instruction tuning in biomedical language processing
Rohanian, O., Nouriborji, M., Kouchaki, S., Nooralahzadeh, F., Clifton, L., & Clifton, D. A. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.103007 · Cited by 46
Shyr, C., Ren, B., Hsu, C.-Y., Nguyen, M.-H., Enger, M., Yan, C., Tinker, R. J., Cassini, T. A., Hamid, R., Wright, A. T., et al. (2026) · Journal of the American Medical Informatics Association · doi:10.1101/2025.08.06.25333170 · Cited by 43
Guthrie, E., Levy, D., & del Carmen, G. (2024) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2024.02.016 · Cited by 22
Liu, F., Zhou, H., Wang, K., Yu, Y., Gao, Y., Sun, Z., Liu, S., Sun, S., Zou, Z., Li, Z., et al. (2025) · Cell Reports Medicine · doi:10.1016/j.xcrm.2025.102056 · Cited by 22
A collaborative large language model for drug analysis
Zhou, H., Liu, F., Wu, J., Zhang, W., Huang, G., Clifton, L., Eyre, D. W., Luo, H., Liu, F., Branson, K., et al. (2025) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01471-z · Cited by 44
Benchmarking large language model-based agent systems for clinical decision tasks
Liu, Y., Carrero, Z. I., Jiang, X., Ferber, D., Wölflein, G., Zhang, L., Jayabalan, S., Lenz, T., Hui, Z., & Kather, J. N. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02443-6 · Cited by 37
Alu, F. F., & Oluwadare, S. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1737532 · Cited by 22
ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemes
Hong, S., Xiao, L., Zhang, X., & Chen, J. (2024) · 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822109 · Cited by 21
Generative Artificial Intelligence in Medicine
Shanmugam, D., Agrawal, M., Movva, R., Chen, I. Y., Ghassemi, M., Jacobs, M. L., & Pierson, E. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-095332 · Cited by 41
A Brief Review on Benchmarking for Large Language Models Evaluation in Healthcare
Cilar, L., Chen, H., Chen, A., Topaz, M., Tam, W., Bian, J., & Štiglic, G. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70010 · Cited by 39
Large language models for bioinformatics
Ruan, W., Lyu, Y., Zhang, J., Cai, J., Shu, P., Yang, G., Lu, Y., Gao, S., Wang, Y., Wang, P., et al. (2025) · Quantitative Biology · doi:10.1002/qub2.70014 · Cited by 48
Agentic AI and the rise of in silico team science in biomedical research
Li, B., Saini, A. K., Hernandez, J. G., & Moore, J. H. (2026) · Nature Biotechnology · doi:10.1038/s41587-026-03035-1 · Cited by 45
Explainable differential diagnosis with dual-inference large language models
Zhou, S., Lin, M., Ding, S., Wang, J., Chen, C., Melton, G. B., Zou, J., & Zhang, R. (2025) · npj Health Systems · doi:10.1038/s44401-025-00015-6 · Cited by 20
A survey on biomedical automatic text summarization with large language models
Huang, Z., Chen, X., Wang, Y., Huang, J., & Zhao, X. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104216 · Cited by 27
SciHorizon: Benchmarking AI-for-Science Readiness from Scientific Data to Large Language Models
Qin, C., Chen, X., Wang, C., Wu, P., Chen, X., Cheng, Y., Zhao, J., Xiao, M., Dong, X., Long, Q., et al. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737403 · Cited by 44
Xu, T., Chen, L., Hu, Z., & Li, B. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127582 · Cited by 30
Adversarial prompt and fine-tuning attacks threaten medical large language models
Yang, Y., Jin, Q., Huang, F., & Lu, Z. (2025) · Nature Communications · doi:10.1038/s41467-025-64062-1 · Cited by 59
GrapeQA: GRaph Augmentation and Pruning to Enhance Question-Answering
Taunk, D., Khanna, L., Kandru, S. V. P. K., Varma, V., Sharma, C., & Tapaswi, M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3587651 · Cited by 49
Applications and Future Prospects of Medical LLMs: A Survey Based on the M-KAT Conceptual Framework
Chang, Y., Yin, J., Li, J., Liu, C., Cao, L., & Lin, S. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02132-5 · Cited by 23
Large Language Models in Integrative Medicine: Progress, Challenges, and Opportunities
Yip, H. F., Li, Z., Zhang, L., & Lü, A. (2025) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.70031 · Cited by 22
Hou, B., Mukherjee, P., Batheja, V., Wang, K. C., Summers, R. M., & Lu, Z. (2025) · Radiology · doi:10.1148/radiol.250617 · Cited by 32
Savage, T., Ma, S. P., Boukil, A., Rangan, E. S., Patel, V., López, I., & Chen, J. H. (2025) · Journal of Medical Internet Research · doi:10.2196/76048 · Cited by 33
Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trends
Lan, Y., Li, X., Du, H., Lu, X., Gao, M. Z., Qian, W., & Zhou, A. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3621181 · Cited by 61
DRLK: Dynamic Hierarchical Reasoning with Language Model and Knowledge Graph for Question Answering
Zhang, M., Dai, R., Dong, M., & He, T. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.342 · Cited by 26
Beyond Accuracy: Investigating Error Types in GPT-4 Responses to USMLE Questions
Roy, S., Khatua, A., Ghoochani, F., Hadler, U., Nejdl, W., & Ganguly, N. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657882 · Cited by 17
Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings
Wolfe, R., Slaughter, I., Han, B., Wen, B., Yang, Y., Rosenblatt, L., Herman, B., Brown, E., Qu, Z., Weber, N., et al. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658966 · Cited by 24
Ali, M., Rao, P. V., Mai, Y., & Xie, B. (2024) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3632620.3671097 · Cited by 33
Easing the cognitive load of general practitioners: AI design principles for future-ready healthcare
Hor, T., Fong, L., Wynne, K., & Verhoeven, B. (2025) · Technovation · doi:10.1016/j.technovation.2025.103208 · Cited by 24
FiTs: Fine-Grained Two-Stage Training for Knowledge-Aware Question Answering
Ye, Q., Cao, B., Chen, N., Xu, W., & Zou, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26629 · Cited by 27
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive BenchmarkPreprint
Liu, F., Li, Z., Zhou, H., Yin, Q., Yang, J., Tang, X., Luo, C., Zeng, M. Y., Jiang, H., Gao, Y., et al. (2024) · medRxiv · doi:10.1101/2024.04.24.24306315 · Cited by 17
When It Comes to Benchmarks, Humans Are the Only Way
Rodman, A. M., Zwaan, L., Olson, A. P. J., & Manrai, A. K. (2025) · NEJM AI · doi:10.1056/aie2500143 · Cited by 24
Sociotechnical Implications of Generative Artificial Intelligence for Information Access
Mitra, B., Cramer, H. S. M., & Gurevich, O. (2024) · The information retrieval series · doi:10.1007/978-3-031-73147-1_7 · Cited by 35
LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models
Yang, H., Chen, H. M., Hui, G., Chen, Y., Lin, C.-S., Hu, S., Hu, J., Wu, X., & Wang, X. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228647 · Cited by 83
Şensoy, E., & Çıtırık, M. (2023) · International Ophthalmology · doi:10.1007/s10792-023-02893-x · Cited by 19
BioPlanner: Automatic Evaluation of LLMs on Protocol Planning in Biology
O’Donoghue, O., Shtedritski, A., Ginger, J., Abboud, R., Ghareeb, A. E., & Rodriques, S. G. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.162 · Cited by 72
Efficient Compressing and Tuning Methods for Large Language Models: A Systematic Literature Review
Kim, G. I., Hwang, S., & Jang, B. (2025) · ACM Computing Surveys · doi:10.1145/3728636 · Cited by 55
Lu, H., Lin, Y., Li, Z., Yiu, M. L., Gao, Y., & Uddin, S. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103216 · Cited by 23
Bentegeac, R., Guellec, B. L., Kuchcinski, G., Amouyel, P., & Hamroun, A. (2025) · Journal of Medical Internet Research · doi:10.2196/64348 · Cited by 40
Wang, D., Ye, J., Li, J., Liang, J., Zhang, Q., Hu, Q., Pan, C., Wang, D., Liu, Z. Y., Shi, W. H., et al. (2025) · JMIR Medical Education · doi:10.2196/70190 · Cited by 18
Coordinated LLM multi-agent systems for collaborative question-answer generation
Saadaoui, S., & Alonso, E. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114627 · Cited by 32
LMGFuse: Language Models and Graph reasoning Fuse deeply for question answering
Wang, A., Duan, P., Li, Y., Hu, W., & Xiong, S. (2023) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2023-023 · Cited by 16
A comparison of chain-of-thought reasoning strategies across datasets and models
Hebenstreit, K., Praas, R., Kiesewetter, L. P., & Samwald, M. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1999 · Cited by 15
Wang, Z., Duan, J., Yuan, C., Chen, Q., Chen, T., Zhang, Y., Wang, R., Shi, X., & Xu, K. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109553 · Cited by 50
Retrieval Augmented Generation for Large Language Models in Healthcare: A Systematic ReviewPreprint
Amugongo, L. M., Mascheroni, P., Brooks, S. G., Doering, S., & Seidel, J. (2024) · Preprints.org · doi:10.20944/preprints202407.0876.v1 · Cited by 15
Automatic Interactive Evaluation for Large Language Models with State Aware Patient SimulatorPreprint
Liao, Y., Meng, Y., Wang, Y., Li, L., Liu, H., Wang, Y., & Wang, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4890649 · Cited by 58
The Medium is the Message: How Non-Clinical Information Shapes Clinical Decisions in LLMs
Gourabathina, A., Gerych, W., Pan, E., & Ghassemi, M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732121 · Cited by 32
Self-citationMedical multimodal large language models: A systematic review
Hu, Y., Xu, C., Lin, B., Yang, W., & Tang, Y. Y. (2025) · Intelligent Oncology · doi:10.1016/j.intonc.2025.09.005 · Cited by 14
Civettini, I., Zappaterra, A., Granelli, B. M., Rindone, G., Aroldi, A., Bonfanti, S., Colombo, F., Fedele, M., Grillo, G., Parma, M., et al. (2023) · British Journal of Haematology · doi:10.1111/bjh.19200 · Cited by 17
Cusumano, G., D'arrigo, S., Terminella, A., & Lococo, F. (2024) · doi:10.20944/preprints202405.0759.v1 · Cited by 14
Fewer Than 1% of Explainable AI Papers Validate Explainability with Humans
Suh, A., Hurley, I., Smith, N., & Siu, H. C. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719964 · Cited by 37
Wang, S., Zhao, F., Bu, D., Lu, Y., Gong, M., Liu, H., Yang, Z., Zeng, X., Yuan, Z., Wan, B., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-64142-2 · Cited by 23
MedSumGraph: enhancing GraphRAG for medical QA with summarization and optimized prompts
Kim, D.-H., Yoo, S., & Jeong, O. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103311 · Cited by 22
Pawar, D., Phansalkar, S., Sharma, A., Sahu, G. K., Ang, C. K., & Lim, W. H. (2023) · Sustainability · doi:10.3390/su15054216 · Cited by 25
Hou, R., Wang, X., Zhang, W., Song, Z., Wang, K., Chen, Y., Liu, J., & Ruan, T. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3529682 · Cited by 18
Yao, Z., Parashar, A., Zhou, H., Jang, W. S., Ouyang, F., Yang, Z., & Yu, H. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.538 · Cited by 27
The Development Landscape of Large Language Models for Biomedical Applications
Cao, Z., Keloth, V. K., Xie, Q., Qian, L., Liu, Y., Wang, Y., Shi, R., Zhou, W., Yang, G. F., Zhang, J., et al. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-102224-074736 · Cited by 21
EvoMDT: a self-evolving multi-agent system for structured clinical decision-making in multi-cancer
Liu, Q., Hu, Z., Huang, T., Niu, Y., Zhang, X., Ma, S., Lin, C., Goh, K. H., Kwon, H. E., Gao, F., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-025-02304-8 · Cited by 31
Graph-Based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey
Zhu, Z., Huang, T., Wang, K., Ye, J., Chen, X., & Luo, S. (2026) · ACM Computing Surveys · doi:10.1145/3795880 · Cited by 60
Mayo‐Yáñez, M., Lechien, J. R., Saibene, A. M., Vaira, L. A., Maniaci, A., & Chiesa-Estomba, C. M. (2024) · Indian Journal of Otolaryngology and Head & Neck Surgery · doi:10.1007/s12070-024-04729-1 · Cited by 17
Hammane, Z., Ben-Bouazza, F.-E., & Fennan, A. (2024) · 2024 International Conference on Intelligent Systems and Computer Vision (ISCV) · doi:10.1109/iscv60512.2024.10620139 · Cited by 11
Unveiling the power of language models in chemical research question answering
Chen, X., Wang, T., Guo, T., Guo, K., Zhou, J., Li, H., Song, Z., Gao, X., & Zhang, X. (2025) · Communications Chemistry · doi:10.1038/s42004-024-01394-x · Cited by 36
Yang, Y., Jin, Q., Zhu, Q., Wang, Z., Álvarez, F. E., Wan, N., Hou, B., & Lu, Z. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-094851 · Cited by 25
AcuGPT-Agent: An LLM-powered intelligent system for acupuncture-based infertility treatment
Wen, J., Liu, D., Xie, Y., Ren, Y., Wang, J., Xia, Y., & Zhu, P. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131116 · Cited by 20
You Don’t Need Labeled Data for Open-Book Question Answering
Gholami, S., & Noori, M. (2021) · Applied Sciences · doi:10.3390/app12010111 · Cited by 22
MEDITRON: Open Medical Foundation Models Adapted for Clinical PracticePreprint
Chen, Z., Romanou, A., Bonnet, A., Hernández-Cano, A., AlKhamissi, B., Matoba, K., Salvi, F., Pagliardini, M., Fan, S., Köpf, A., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4139743/v1 · Cited by 31
Agentic AI in Healthcare: Opportunities, Challenges, and Future Directions
Gridach, M., Nanavati, J., Zine El Abidine, K., Yacoubian, C., & Mack, C. (2026) · ACM Computing Surveys · doi:10.1145/3809164 · Cited by 8
AgentClinic: a multimodal benchmark for tool-using clinical AI agents
Schmidgall, S., Ziaei, R., Harris, C., Kim, J. W., Reis, E. P., Jopling, J., & Moor, M. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02674-7 · Cited by 344
Generative artificial intelligence for literature reviews
Wagner, G., Prester, J., Mousavi, R., Lukyanenko, R., & Paré, G. (2026) · Journal of Information Technology · doi:10.1177/02683962261425675 · Cited by 20
Cusumano, G., D'arrigo, S., Terminella, A., & Lococo, F. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13133750 · Cited by 16
Wang, B., Liu, Y., Tian, H., Hua, R., Chang, K., Xia, J., Dai, X., Gao, Z., Liu, S., Wang, R., et al. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2024.9010125 · Cited by 14
PodGPT: an audio-augmented large language model for research and education
Jia, S., Bit, S., Searls, E., Lauber, M. V., Fan, P., Wang, W. M., Claus, L. A., Jasodanand, V. H., Veerapaneni, D., Au, R., et al. (2025) · npj Biomedical Innovations. · doi:10.1038/s44385-025-00022-0 · Cited by 15
Large Language Model Uncertainty Measurement and Calibration for Medical Diagnosis and Treatment
Savage, T., Wang, J., Gallo, R., Boukil, A., Patel, V., Ahmad Safavi-Naini, S. A., Soroush, A., & Chen, J. H. (2024) · doi:10.1101/2024.06.06.24308399 · Cited by 11
LongHealth: A Question Answering Benchmark with Long Clinical Documents
Adams, L., Busch, F., Han, T., Excoffier, J.-B., Ortala, M., Löser, A., Aerts, H. J., Kather, J. N., Truhn, D., & Bressem, K. (2024) · arXiv · doi:10.48550/arxiv.2401.14490 · Cited by 10
Metadata
- Authors
- Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P.
- Affiliations
- Massachusetts Institute of Technology (US)
- Huazhong University of Science and Technology (CN)
- Computer Science and Artificial Intelligence, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
- Corresponding author
- Jin, Di
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2021-07-12
- Volume
- 11
- Issue
- 14
- Pages
- 6421
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3162922479
- Cited by
- 718
- References
- 41
- Citations by year
- 2026: 285 · 2025: 241 · 2024: 120 · 2023: 51 · 2022: 12 · 2021: 9
- FWCI
- 16.96
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- Preprints.org (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- open-domain question answering, medical question answering, multiple-choice questions, Chinese language, document retrieval, machine reading comprehension, disease diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.