PubMedQA: A Dataset for Biomedical Research Question Answering
Open accessThe DOI matches its Crossref record; 24 of the 26 references it cites were found in a registry or an index.
Abstract
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, Xinghua Lu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“given that medical question datasets such as PubMedQA42 and MedQA43 consist of fixed question sets susceptible to memorization”
From An automated framework for assessing how well LLMs cite relevant medical references (2025) · cites this paper as [43]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“All analyses were performed on publicly available, de-identified benchmark datasets—MIRAGE [23], PubMedQA [24], and the medical licensing examination questions of Katz et al. [25]—used in accordance with their respective licenses and terms of use.”
From The economics of accuracy for medical reasoning with large language models (2026) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“PubMedQA (Jin et al., 2019) is a biomedical research QA dataset under the MIT license. It contains 1,000 expert-annotated questions ( pqa\_labeled\_split ) and 211K machine-labeled questions from PubMed abstracts (the most widely used biomedical literature resource).”
From MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025) · cites this paper as Jin et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Notably, in healthcare, medical NLP benchmarks like MedQA7, PubMedQA8 and the Massive Multitask Language Understanding (MMLU) serve as the de-facto reporting standard for state-of-the-art medical LLMs9–11 with claims of ‘superhuman’ performance in patient-facing tasks12.”
From Medical large language models are vulnerable to data-poisoning attacks (2025) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“The ground truth question-answer pairs are sourced from the pqa\_labeled split of PubMedQA (Jin et al., 2019), which contains 1,000 expert-annotated samples, supplemented with 9,000 instances randomly selected from the pqa\_artificial split.”
From MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025) · cites this paper as Jin et al., 2019
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 2×
“We fine-tune BioBERT (Lee et al., 2019) on Pub-MedQA as a baseline.”
MethodThe Stanford CoreNLP Natural Language Processing Toolkit
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014) · Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.3115/v1/p14-5010 · Cited by 7,037
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“Unlike other QA datasets in which questions are asked by crowd-workers for existing contexts (Rajpurkar et al., 2016; Yang et al., 2018; Kočiskỳ et al., 2018), in PubMedQA contexts are generated to answer the questions and both are written by the same authors.”
BackgroundNatural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 6×
“, 2018), Natural Questions (Kwiatkowski et al., 2019), ShARC (Saeidi et al.”
BackgroundMethodInfluentialBoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1300 · Cited by 1,855
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 6×
“Supporting fact for the answer is highlighted . et al., 2018; Yang et al., 2018; Kwiatkowski et al., 2019), the largest annotated biomedical QA dataset, BioASQ (Tsatsaronis et al., 2015) has less than 3k training instances, most of which are simple factual questions.”
BackgroundMethodInfluentialSelf-citationEnhanced LSTM for Natural Language Inference
Chen, Q., Zhu, X., Ling, Z.-H., Wei, S., Jiang, H., & Inkpen, D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1152 · Cited by 1,113
Research works printed but not linked: 1
- 2012 Conference and Labs of the Evaluation ForumQuestion Answering For Machine Reading Evaluation (QA4MRE), Rome/Forner, J.[edit.]; ea , pages 1-14.
Cited by
Items 601–700 of 1,026
Clinical Dialogue Transcription Error Correction Using Seq2Seq Models
Nanayakkara, G., Wiratunga, N., Corsar, D., Martin, K., & Wijekoon, A. (2022) · Studies in computational intelligence · doi:10.1007/978-3-031-14771-5_4 · Cited by 11
Selective UMLS knowledge infusion for biomedical question answering
Park, H., Son, J., Min, J., & Choi, J. (2023) · Scientific Reports · doi:10.1038/s41598-023-41423-8 · Cited by 11
Empowerment of Large Language Models in Psychological Counseling through Prompt Engineering
Huang, S., Fu, F., Yang, K. Q., Zhang, K., & Yang, F. (2024) · IEEE International Conference on Software Engineering and Artificial Intelligence (SEAI) · doi:10.1109/seai62072.2024.10674052 · Cited by 11
Towards Efficient Methods in Medical Question Answering using Knowledge Graph Embeddings
Sengupta, S., Heaton, C., Cui, S., Sarkar, S., & Mitra, P. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821824 · Cited by 11
PediaBench: a comprehensive Chinese pediatric dataset for benchmarking large language models
Zhang, Q., Chen, P., Feng, L., Liu, S., Li, J., Zhao, H., Chen, M., Li, H., & Wang, Y. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-025-41345-w · Cited by 11
CARE-MI: Chinese Benchmark for Misinformation Evaluation in Maternity and Infant Care
Xiang, T., Li, L., Li, L., Bai, M., Wei, L., Wang, B., & García, N. (2023) · neural information processing systems · doi:10.52202/075280-1836 · Cited by 11
Consumer Health Question Answering Using Off-the-Shelf Components
Pugachev, A., Artemova, E., Bondarenko, A., & Braslavski, P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28238-6_48 · Cited by 11
Prompt Discriminative Language Models for Domain Adaptation
Lu, K., Potash, P., Lin, X., Sun, Y., Qian, Z., Zheng, Y. G., Naumann, T., Cai, T., & Lu, J. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.30 · Cited by 11
Few-shot Query-Focused Summarization with Prefix-Merging
Yuan, R., Wang, Z., Cao, Z., & Li, W. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.243 · Cited by 11
Igarashi, Y., Nakahara, K., Norii, T., Miyake, N., Tagami, T., & Yokobori, S. (2024) · Journal of Nippon Medical School · doi:10.1272/jnms.jnms.2024_91-205 · Cited by 11
Biomedical Question Answering: A Comprehensive Review.Preprint
Jin, Q., Yuan, Z. P., Xiong, G., Yu, Q., Tan, C., Chen, M., Huang, S., Liu, X., & Yu, S. (2021) · arXiv (Cornell University) · Cited by 11
Self-citationElectronic Circuit Principles of Large Language ModelsPreprint
Chen, Q., Qin, L., Liu, J., Peng, D., Wang, J., Hu, M., Chen, Z., Che, W., & Liu, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.03325 · Cited by 11
Kawakami, W., Suzuki, K., & Iwasawa, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.18080 · Cited by 11
Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning
Gloaguen, T., Vero, M., Staab, R., & Vechev, M. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2505.16567 · Cited by 11
Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
Jin, R., Shao, P., Wen, Z., Wu, J., Feng, M., Yang, S., Zhang, C. Y., & Tao, J. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2602.01064 · Cited by 11
LongHealth: A Question Answering Benchmark with Long Clinical Documents
Adams, L., Busch, F., Han, T., Excoffier, J.-B., Ortala, M., Löser, A., Aerts, H. J., Kather, J. N., Truhn, D., & Bressem, K. (2024) · arXiv · doi:10.48550/arxiv.2401.14490 · Cited by 10
“PubMedQA’s yes/no/maybe format, while useful, does not provide the nuance required for complex clinical decision-making [7].”
MAGE: Machine-generated Text Detection in the Wild
Li, Y., Li, Q., Cui, L., Bi, W., Wang, Z., Wang, L., Yang, L., Shi, S., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.13242 · Cited by 10
Rani, M., Mishra, B. K., Thakker, D., Babar, M. A., Jones, W., & Din, A. (2024) · International Conference on IT and Industrial Technologies (ICIT) · doi:10.1109/icit63607.2024.10859641 · Cited by 10
Ahn, J., Kang, B. G., Chang, M. Y., & Yoon, S. (2025) · Clinical and Experimental Otorhinolaryngology · doi:10.21053/ceo.2025-00121 · Cited by 10
Foundation Model for Medical Imaging: A Comprehensive Review
Jiao, L., Yang, J., Li, R., Liu, F., Liu, X., Chen, P., Guo, Y., Li, L., Shang, R., & Ma, W. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3618796 · Cited by 11
PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models
Molfetta, L., Frisoni, G., Monaldini, N., & Moro, G. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.507 · Cited by 11
Variational Uncertainty Decomposition for In-Context Learning
Jayasekera, I., Si, J., Valdettaro, F., Chen, W., Faisal, A. A., & Li, Y. (2025) · neural information processing systems · doi:10.52202/085713-0080 · Cited by 10
Flathers, M., Dwyer, B., Rozenblit, E., & Torous, J. B. (2025) · Journal of Psychiatric Practice · doi:10.1097/pra.0000000000000892 · Cited by 10
Khullar, M., Desai, U., Malviya, P., Dalmia, A., & Shi, Z. R. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-industry.144 · Cited by 10
MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation
Ji, Y., Kwak, M. G., Zhang, H., Wu, X., Li, C., & Wang, Y. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.399 · Cited by 10
A benchmark for evaluating diagnostic questioning efficiency of LLMs in patient conversations
Werthaim, M., Kimhi, M., Apartsin, A., & Aperstein, Y. (2026) · Scientific Reports · doi:10.1038/s41598-026-37022-y · Cited by 10
Liu, Z., Huang, P., Xu, Z., Li, X., Liu, S., Peng, C., Xin, H., Yan, Y., Wang, S., Han, X., et al. (2026) · AI Open · doi:10.1016/j.aiopen.2026.02.002 · Cited by 12
Self-citationA short survey on small reasoning models: training, inference, applications, and research directions
Wang, C., Zhang, T., Hong, R., & Huang, J. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-025-50990-0 · Cited by 10
Rogoz, A.-C., Ionescu, R. T., Anghel, A.-V., Antone-Iordache, I.-L., Coniac, S., & Ionescu, A.-I. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02465-0 · Cited by 10
RAG-X: Systematic Diagnosis of Retrieval-Augmented Generation for Medical Question Answering
Sivakumar, A., Sugumaran, V., & Qiang, Y. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3839903 · Cited by 10
Wang, X., Zong, Y., Zhou, X., Xu, L., He, W., & Quan, S. (2024) · The Journal of Physical Chemistry B · doi:10.1021/acs.jpcb.3c06526 · Cited by 10
Efficient Medical Question Answering with Knowledge-Augmented Question Generation
Khlaut, J., Dancette, C., Ferreres, É., Benani, A., Herent, H., & Manceron, P. (2024) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2024.clinicalnlp-1.2 · Cited by 10
Jadhav, S., Shanbhag, A., Joshi, S., Date, A., & Sonawane, S. S. (2024) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2024.clinicalnlp-1.36 · Cited by 10
Infusing Multi-Hop Medical Knowledge Into Smaller Language Models for Biomedical Question Answering
Chen, J., Wei, Z., Shen, W., & Shang, R. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3547444 · Cited by 10
Tran, H., Wang, J., Ting, Y., Yu, H., Huang, W., & Chen, T. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-industry.23 · Cited by 10
Shh, don't say that! Domain Certification in LLMs
Emde, C., Paren, A., Arvind, P., Kayser, M., Rainforth, T., Lukasiewicz, T., Ghanem, B., Torr, P. H. S., & Bibi, A. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2502.19320 · Cited by 11
Zhang, K., Feng, Y., & Zheng, J. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3ib59n936jrz · Cited by 10
Cross-document attention-based gated fusion network for automated medical licensing exam
Liu, J., Ren, J., Lu, Z., He, W., Cui, M., Zhang, Z., & Bai, R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117588 · Cited by 10
qaAskeR$$^+$$: a novel testing method for question answering software via asking recursive questions
Xie, X., Jin, S., & Chen, S. (2023) · Automated Software Engineering · doi:10.1007/s10515-023-00380-2 · Cited by 10
Research on a massively large artificial intelligence model and its application in medicine
Guo, H., Liu, P., Lu, R., Yang, F., Xu, H., Zhuang, Y., Huang, G. M., Song, S., & He, K. (2024) · Scientia Sinica Vitae · doi:10.1360/ssv-2022-0298 · Cited by 10
CoverBench: A Challenging Benchmark for Complex Claim VerificationPreprint
Jacovi, A., Ambar, M., Ben-David, E., Shaham, U., Feder, A., Geva, M., Marcus, D., & Caciularu, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.03325 · Cited by 10
He, J., Fan, J., Jiang, B., Hounie, I., Roth, D., & Ribeiro, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.15062 · Cited by 10
MiraMind: Benchmarking Reliable Mental Health Reasoning beyond Answer AccuracyPreprint
Xiao, M., Yang, K., Zhao, P., Zhang, E., Kuang, Z., Liu, Z., Han, W., Liao, S., Huang, L., Xiong, G., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.09636 · Cited by 10
Self-citationEnabling On-Device Medical AI Assistants via Input-Driven Saliency AdaptationPreprint
Kallakurik, U., Humes, E., Jonna, R., Lin, X., & Mohsenin, T. (2025) · arXiv (Cornell University) · doi:10.13140/rg.2.2.10530.34244 · Cited by 10
On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A SurveyPreprint
Zhang, M., Zhang, X., Zhao, X., Huang, S., Hu, B., & Zhang, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.20783 · Cited by 10
What Language Model to Train if You Have One Million GPU Hours?
Scao, T. L., Wang, T., Hesslow, D., Saulnier, L., Bekman, S., Bari, M. S., Biderman, S., Elsahar, H., Muennighoff, N., Phang, J., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.15424 · Cited by 9
Fact retrieval from knowledge graphs through semantic and contextual attention
Chaudhary, A., Rajabi, E., Kafaie, S., & Milios, E. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127612 · Cited by 9
LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations
Vujanic, R., & Rückstieß, T. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.2008 · Cited by 9
M3Retrieve: Benchmarking Multimodal Retrieval for Medicine
Acharya, A., Ghosh, A., Verma, P., Pasupa, K., Saha, S., & Singh, P. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.771 · Cited by 9
Large language models in healthcare: a systematic evaluation on medical Q/A datasets
Tofeeq, K., Naseer, A., & Wali, A. (2025) · Health Information Science and Systems · doi:10.1007/s13755-025-00397-9 · Cited by 9
Reasoning LLMs in the Medical Domain: A Literature Survey
Berger, A., Khanna, S., Sparrenberg, L., Deußer, T., Berghaus, D., & Sifa, R. (2025) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa65442.2025.11247922 · Cited by 9
Advances, Evaluation, and Explainability of Large Language Models in Healthcare: A Systematic Review
Amin, S. U., Guizani, M., & Hossain, M. S. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3786334 · Cited by 10
RxSafeBench: Identifying Medication Safety Issues of Large Language Models in Simulated Consultation
Zhao, J., Xu, L., Tan, M., Zhang, L., Argha, A., Alinejad-Rokny, H., & Yang, M. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356353 · Cited by 9
Large language models in healthcare and biomedical informatics: A comprehensive review
Hornback, A. M., Sathu, H., Kim, K., Wang, Y., Zhu, Y., Isgut, M., Avula, P., Khimani, A., & Wang, M. D. (2026) · Innovation and Emerging Technologies · doi:10.1142/s2737599426300011 · Cited by 9
Bi, Z., Chen, L., Luo, H., Song, J., Ge, E., Huang, J., Wang, T., Chen, K., Chia, X. X., Wei, Z., et al. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105025 · Cited by 9
ReMedQA: Are We Done With Medical Multiple-Choice Benchmarks?
Cocchieri, A., Ragazzi, L., Tagliavini, G., & Moro, G. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.124 · Cited by 10
Predict the Retrieval! Test Time Adaptation for Retrieval Augmented Generation
Sun, X., Chen, Z., Liu, Q., Wu, S., Song, B., Wang, W., Wang, Z., & Wang, L. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464478 · Cited by 9
Security and Privacy of Large Language Models: Threat Taxonomy, Ethical Implications, and Governance
Pribisalić, M., & Martinčić-Ipšić, S. (2026) · AI · doi:10.3390/ai7050152 · Cited by 9
Medical Reasoning With Large Language Models: A Systematic Review and Evaluation
Ren, X., Fan, C., Ma, W., He, H., Gao, C., Zhao, X., & Feng, F. (2026) · iNew Medicine · doi:10.1002/inm3.70056 · Cited by 9
Privacy-Aware Distributed RAG with Confidence-Aware Retrieval Routing
Huang, W., Xu, S., & Jiang, Z. (2026) · International Conference on Image Processing and Artificial Intelligence (ICIPAI) · doi:10.1109/icipai70034.2026.11605698 · Cited by 9
Alonso-Sánchez, M. F., Rivera, L. Z., Otero, M., Portal, J., Álvaro Cavieres, F., & Alfaro, P. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1244694 · Cited by 9
Large Language Models for Healthcare: Applications, Models, Datasets, and Challenges
Rezgui, K. (2024) · International Conference on Control, Decision and Information Technologies (CoDIT) · doi:10.1109/codit62066.2024.10708253 · Cited by 9
Knowledge Hierarchy Guided Biological-Medical Dataset Distillation for Domain LLM Training
Cai, X., Wang, C., Long, Q., Zhou, Y., & Xiao, M. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-3827-0_17 · Cited by 9
Park, J.-H., Cheong, C., Kang, S., Lee, I., Lee, S., Yoon, K. S., & Lee�, H. (2024) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi62660.2024.10913702 · Cited by 9
Pham, T.-H., Bui, T.-D., Quang, M. L., Pham, T.-H., Ngo, C., & Hy, T. S. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00281 · Cited by 9
ENTERPRISE LARGE LANGUAGE MODEL EVALUATION BENCHMARK
Wang, L., Yi, D., Jose, D., Passarelli, J. V., Gao, J., Leventis, J., & Li, K. (2025) · International Conference on Machine Learning Techniques and NLP (MLNLP) · doi:10.5121/csit.2025.152001 · Cited by 9
Ke, X., Deng, H., Liu, X., Rao, J., Song, Z., Yu, J., & Zhang, M. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.293 · Cited by 10
Attention-based aspect reasoning for knowledge base question answering on clinical notes
Wang, P., Shi, T., Agarwal, K., Choudhury, S., & Reddy, C. K. A. (2022) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3535508.3545518 · Cited by 9
Experiments on Portuguese Clinical Question Answering
Oliveira, L. E. S. E., Schneider, E. T. R., Gumiel, Y. B., da Luz, M. A. P., Paraíso, E. C., & Moro, C. M. C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-91699-2_10 · Cited by 9
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERTPreprint
Meng, Z., Liu, F., Clark, T. H., Shareghi, E., & Collier, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.04810 · Cited by 9
Building a literature knowledge base towards transparent biomedical AIPreprint
Huang, Y., Han, Z., Luo, X., Luo, X., Gao, Y., Zhao, M., Tang, F., Wang, Y., Chen, J., Li, C., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.09.22.614323 · Cited by 9
Self-citationQuarkMed Medical Foundation Model Technical ReportPreprint
Li, A., Yan, B., Cai, B., Li, C., Zhao, C., Yao, F., Liu, G., Jiang, G., Xu, J., Liang, D., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.11894 · Cited by 9
Benchmarking GPT-5 for biomedical natural language processingPreprint
Hou, Y., Zhan, Z., Zeng, M., Wu, Y., Zhou, S., & Zhang, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.04462 · Cited by 9
G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledge
Luo, L., Zhao, Z., Liu, J., Qiu, Z., Dong, J., Panev, S., Gong, C., Vu, T.-T., Haffari, G., Phung, D., et al. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2509.24276 · Cited by 9
MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answering
Hao, Y., Alhamoud, K., Jeong, H., Zhang, H., Puri, I., Torr, P., Schaekermann, M., Stern, A. D., & Ghassemi, M. (2025) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.2505.24040 · Cited by 10
Wong, H. M., Heesen, P., Janetzky, P., Bendszus, M., & Feuerriegel, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.17308 · Cited by 9
Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?Preprint
Li, K., He, Y., & Li, Q. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33209 · Cited by 10
An Empirical Evaluation of Large Language Models on Consumer Health Questions
Abrar, M., Sermet, Y., & Demir, I. (2025) · arXiv · doi:10.48550/arxiv.2501.00208 · Cited by 8
“This study AskDocs subreddit: MedRedQA Large ( n = 5000) GPT4-o, Mistral-123B, Mistral-7B, Llama 3.1-70B, and Gemini Flash are tested on a large number of consumer health questions and evaluated by LLMs.”
InfluentialText Fluoroscopy: Detecting LLM-Generated Text through Intrinsic Features
Yu, X., Chen, K., Yang, Q., Zhang, W., & Yu, N. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.885 · Cited by 8
“Specifically, the three datasets are Xsum (Narayan et al., 2018) for news articles, WritingPrompts (Fan et al., 2018) for story writing, PubMedQA (Jin et al., 2019) for biomedical research question answering, which are consistent with previous work (Bao et al.) in the field.”
ResultDocLens: Multi-aspect Fine-grained Medical Text Evaluation
Xie, Y., Zhang, S., Cheng, H., Liu, P., Gero, Z., Wong, C., Naumann, T., Poon, H., & Rose, C. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.39 · Cited by 8
Zero-Shot Detection of LLM-Generated Text using Token Cohesiveness
Ma, S., & Wang, Q. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.971 · Cited by 8
Agentic AI in Healthcare: Opportunities, Challenges, and Future Directions
Gridach, M., Nanavati, J., Zine El Abidine, K., Yacoubian, C., & Mack, C. (2026) · ACM Computing Surveys · doi:10.1145/3809164 · Cited by 8
Compound-QA: A Benchmark for Evaluating LLMs on Compound Questions
Hou, Y. T., Luo, Y., Ruan, Z., Wang, H., Ge, W., Chen, Y., & Chen, G. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464025 · Cited by 8
HumanMoD: A multi-RAG collaborative LLM for inclusive urban public healthcare services
Sun, S. M., Zhong, Z., Yu, N., Gong, X., & Yang, K. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113684 · Cited by 8
Zhang, X., Li, L., Zhou, Y., Zhou, X. L., Zhang, Y., & Wu, X. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1218 · Cited by 8
MPIB: A Benchmark for Medical Prompt Injection Attacks and Clinical Safety in LLMs
Lee, J., Jang, H., & Choi, K. S. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1670 · Cited by 8
Xiao, Z., Zhang, R., Feng, Y., Zhu, L., Peng, L., & Yu, L. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00124 · Cited by 8
MedMT-Bench: Can LLMs Memorize and Understand Long Multi-Turn Conversations in Medical Scenarios?
Yang, L., Yang, Y., Wang, X., Liu, C., & Yang, H. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.762 · Cited by 8
Watanabe, Y., Kobashi, Y., Kojima, T., Iwasawa, Y., Okuno, Y., & Matsuo, Y. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-industry.47 · Cited by 8
Chen, W., Huang, G., Wang, W., & Zhu, Z. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1847 · Cited by 8
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models
Thangarasa, V., Salem, M. A., Saxena, S., Leong, C.-Y., Hestness, J., & Lie, S. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.14 · Cited by 8
MeRino: Entropy-Driven Design for Generative Language Models on IoT Devices
Zhao, Y., Lin, M., Tang, H., Wu, Q., & Wang, J. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i21.34445 · Cited by 8
SPARC: Subspace-Aware Prompt Adaptation for Robust Continual Learning in LLMs
Jayasuriya, D., Tayebati, S., Ettori, D., Krishnan, R., & Trivedi, A. R. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228226 · Cited by 8
LM-Based Word Embeddings Improve Biomedical Named Entity Recognition: A Detailed Analysis
Akhtyamova, L., & Cardiff, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-45385-5_56 · Cited by 8
Xu, F., Liu, J., Lin, Q., Zhao, T., Zhang, J., & Zhang, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3317254 · Cited by 8
Designing Retrieval-Augmented Language Models for Clinical Decision Support
Quigley, K., Koker, T., Taylor, J., Mancuso, V., & Brattain, L. J. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-63592-2_13 · Cited by 8
Beyond fine-tuning: LoRA modules boost near-OOD detection and LLM security
Salimbeni, E., Craighero, F., Khasanova, R., Vasic, M., & Vandergheynst, P. (2024) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 8
Zhu, J., Xu, A., Tan, M., & Yang, M. (2024) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/2024.clpsych-1.23 · Cited by 8
Carrami, E. M., & Sharifzadeh, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.13653 · Cited by 8
MoDEM: Mixture of Domain Expert ModelsPreprint
Simonds, T., Kurniawan, K., & Lau, J. H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.07490 · Cited by 8
Metadata
- Authors
- Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X.
- Affiliations
- University of Pittsburgh (US)
- Carnegie Mellon University (US)
- Google (United States) (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 2567-2577
- OpenAlex
- W2970482702
- Cited by
- 685
- References
- 26
- Citations by year
- 2026: 188 · 2025: 215 · 2024: 107 · 2023: 75 · 2022: 36 · 2021: 39 · 2020: 25
- FWCI
- 75.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- question answering, biomedical research, PubMed, scientific publications, datasets, biomedical question answering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.