PubMedQA: A Dataset for Biomedical Research Question Answering
Open accessThe DOI matches its Crossref record; 24 of the 26 references it cites were found in a registry or an index.
Abstract
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, Xinghua Lu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“given that medical question datasets such as PubMedQA42 and MedQA43 consist of fixed question sets susceptible to memorization”
From An automated framework for assessing how well LLMs cite relevant medical references (2025) · cites this paper as [43]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“All analyses were performed on publicly available, de-identified benchmark datasets—MIRAGE [23], PubMedQA [24], and the medical licensing examination questions of Katz et al. [25]—used in accordance with their respective licenses and terms of use.”
From The economics of accuracy for medical reasoning with large language models (2026) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“PubMedQA (Jin et al., 2019) is a biomedical research QA dataset under the MIT license. It contains 1,000 expert-annotated questions ( pqa\_labeled\_split ) and 211K machine-labeled questions from PubMed abstracts (the most widely used biomedical literature resource).”
From MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025) · cites this paper as Jin et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Notably, in healthcare, medical NLP benchmarks like MedQA7, PubMedQA8 and the Massive Multitask Language Understanding (MMLU) serve as the de-facto reporting standard for state-of-the-art medical LLMs9–11 with claims of ‘superhuman’ performance in patient-facing tasks12.”
From Medical large language models are vulnerable to data-poisoning attacks (2025) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“The ground truth question-answer pairs are sourced from the pqa\_labeled split of PubMedQA (Jin et al., 2019), which contains 1,000 expert-annotated samples, supplemented with 9,000 instances randomly selected from the pqa\_artificial split.”
From MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025) · cites this paper as Jin et al., 2019
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 2×
“We fine-tune BioBERT (Lee et al., 2019) on Pub-MedQA as a baseline.”
MethodThe Stanford CoreNLP Natural Language Processing Toolkit
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014) · Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.3115/v1/p14-5010 · Cited by 7,037
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“Unlike other QA datasets in which questions are asked by crowd-workers for existing contexts (Rajpurkar et al., 2016; Yang et al., 2018; Kočiskỳ et al., 2018), in PubMedQA contexts are generated to answer the questions and both are written by the same authors.”
BackgroundNatural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 6×
“, 2018), Natural Questions (Kwiatkowski et al., 2019), ShARC (Saeidi et al.”
BackgroundMethodInfluentialBoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1300 · Cited by 1,855
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 6×
“Supporting fact for the answer is highlighted . et al., 2018; Yang et al., 2018; Kwiatkowski et al., 2019), the largest annotated biomedical QA dataset, BioASQ (Tsatsaronis et al., 2015) has less than 3k training instances, most of which are simple factual questions.”
BackgroundMethodInfluentialSelf-citationEnhanced LSTM for Natural Language Inference
Chen, Q., Zhu, X., Ling, Z.-H., Wei, S., Jiang, H., & Inkpen, D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1152 · Cited by 1,113
Research works printed but not linked: 1
- 2012 Conference and Labs of the Evaluation ForumQuestion Answering For Machine Reading Evaluation (QA4MRE), Rome/Forner, J.[edit.]; ea , pages 1-14.
Cited by
Items 401–500 of 1,026
Language Models Improve When Pretraining Data Matches Target TasksPreprint
Mizrahi, D., Larsen, A., Allardice, J. R., Petryk, S., Gorokhov, Y., Li, J. J.-P., Fang, A., Gardner, J., Gunter, T., & Dehghan, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.12466 · Cited by 23
LLM Fingerprinting via Semantically Conditioned Watermarks
Gloaguen, T., Staab, R., Jovanović, N. M., & Vechev, M. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2505.16723 · Cited by 23
Alu, F. F., & Oluwadare, S. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1737532 · Cited by 22
A Survey on the Optimization of Large Language Model-based Agents
Du, S., Zhao, J., Shi, J., Xie, Z., Jiang, X., Bai, Y., & He, L. (2025) · arXiv · doi:10.48550/arxiv.2503.12434 · Cited by 22
Liu, F., Zhou, H., Wang, K., Yu, Y., Gao, Y., Sun, Z., Liu, S., Sun, S., Zou, Z., Li, Z., et al. (2025) · Cell Reports Medicine · doi:10.1016/j.xcrm.2025.102056 · Cited by 22
“MetaGP was refined with a focused dataset that included approximately 630,000 EHR records for rare diseases and emergency conditions; multimodal imaging data comprising over 600,000 chest X-rays (CXRs) and nearly 24,000 computed tomography (CT) scans; medical QA questions (Table 1) sourced from established databases su…”
Guthrie, E., Levy, D., & del Carmen, G. (2024) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2024.02.016 · Cited by 22
Liu, S., Li, X., Liu, Z., Yan, Y., Yang, C., Zeng, Z., Liu, Z., Sun, M., & Ge, Y. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.301 · Cited by 23
Self-citationImproving Health Question Answering with Reliable and Time-Aware Evidence Retrieval
Vladika, J., & Matthes, F. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.295 · Cited by 22
Hammane, Z., Ben-Bouazza, F.-E., & Fennan, A. (2024) · 2024 International Conference on Intelligent Systems and Computer Vision (ISCV) · doi:10.1109/iscv60512.2024.10620139 · Cited by 11
Adaptive Compressed-Based Privacy-Preserving Large Language Model for Sensitive Healthcare
Gong, X., Gao, J., Sun, S. M., Zhong, Z., Shi, Y., Zeng, H., & Yang, K. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3558935 · Cited by 22
AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark
Chen, J., Wang, N., Li, C., Wang, B., Xiao, S., Xiao, H., Liao, H., Lian, D., & Liu, Z. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.982 · Cited by 22
Self-citationDALD: Improving Logits-based Detector without Logits from Black-box LLMs
Zeng, C., Tang, S., Yang, X., Chen, Y., Sun, Y., Xu, Z., Li, Y., Chen, H., Cheng, W., & Xu, D. (2024) · neural information processing systems · doi:10.52202/079017-1743 · Cited by 22
From Beginner to Expert: Modeling Medical Knowledge into General LLMsPreprint
Li, Q., Yang, X., Wang, H., Wang, Q., Liu, L., Wang, J., Zhang, Y., Chu, M., Hu, S., Chen, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.01040 · Cited by 23
OpenResearcher: Unleashing AI for Accelerated Scientific ResearchPreprint
Zheng, Y.-X., Sun, S., Qiu, L., Ru, D., Jiayang, C., Li, X.-F., Lin, J.-F., Wang, B.-J., Luo, Y., Pan, R., et al. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2408.06941 · Cited by 31
Know Your RAG: Dataset Taxonomy and Generation Strategies for Evaluating RAG SystemsPreprint
Lima, R. T. D., Gupta, S., Berrospi, C., Mishra, L., Dolfi, M., Staar, P., & Vagenas, P. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.19710 · Cited by 23
HealthFlow: A Self-Evolving AI Agent with Meta Planning for Autonomous Healthcare ResearchPreprint
Zhu, Y., Qi, Y., Wang, Z., Gu, L., Sui, D., Hu, H., Zhang, X., He, Z., He, J., Ma, L., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.02621 · Cited by 23
F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual WorldPreprint
Zhang, Z., Liao, Z.-H., Yu, H., Di, P., & Wang, R. (2026) · arXiv.org · doi:10.48550/arxiv.2603.19223 · Cited by 17
ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemes
Hong, S., Xiao, L., Zhang, X., & Chen, J. (2024) · 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822109 · Cited by 21
“To better align with real-world application scenarios, our study focused on a zero-shot setting.”
InfluentialThe Development Landscape of Large Language Models for Biomedical Applications
Cao, Z., Keloth, V. K., Xie, Q., Qian, L., Liu, Y., Wang, Y., Shi, R., Zhou, W., Yang, G. F., Zhang, J., et al. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-102224-074736 · Cited by 21
Yu, X., Yu, Y., Liu, D., Chen, K., Zhang, W., Yu, N., & Shao, J. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.754 · Cited by 22
Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting
Diao, M., Yang, L., Gong, W., Zhang, Y., Yan, Z., Han, Y., Liang, K., Xu, W., & Ma, Z. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.703 · Cited by 21
QuerySum: A Multi-Document Query-Focused Summarization Dataset Augmented with Similar Query Clusters
Liu, Y., Wang, Z., & Yuan, R. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29836 · Cited by 21
Impact of high-quality, mixed-domain data on the performance of medical language models
Griot, M., Hemptinne, C., Vanderdonckt, J. M., & Yüksel, D. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae120 · Cited by 21
An Empirical Study of Retrieval Augmented Generation with Chain-of-Thought
Zhao, Y., Cao, H., Zhao, X., & Ou, Z. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800207 · Cited by 21
Liu, Y., Li, X., Luo, Y., Du, J., Zhang, Y., Lv, T., Yin, H., Tang, X., & Liu, H. (2025) · Engineering · doi:10.1016/j.eng.2025.02.010 · Cited by 22
Way to Specialist: Closing Loop Between Specialized LLM and Evolving Domain Knowledge Graph
Zhang, Y., Chen, L., Li, S., Cao, N., Shi, Y., Ding, J., Qu, Z., Zhou, P., & Bai, Y. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709187 · Cited by 21
Data Mixing Optimization for Supervised Fine-Tuning of Large Language ModelsPreprint
Li, Y., Liu, Z., & Xing, E. P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.11953 · Cited by 21
Self-citationKnowledge or Reasoning? A Close Look at How LLMs Think Across DomainsPreprint
Wu, J., Liu, S., Tu, H., Yu, H., Huang, X., Zou, J., Xie, C., & Zhou, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.02126 · Cited by 21
Agentic Abstention: Do Agents Know When to Stop Instead of Act?Preprint
Luo, H., Wen, B., & Wang, L. L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.28733 · Cited by 23
Jev for Scientific Decisions: Evaluating Semantic Choices and Their ConsequencesPreprint
Deng, B., Fan, S., Zhang, H., & Xie, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.24965 · Cited by 21
Permissioned LLMs: Enforcing Access Control in Large Language Models
Jayaraman, B., Marathe, V. J., Mozaffari, H., Shen, W., & Kenthapadi, K. (2025) · neural information processing systems · doi:10.52202/085713-2731 · Cited by 20
SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems
Bodea, A.-E., Meisenbacher, S., Klymenko, A., & Matthes, F. (2026) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml68715.2026.00039 · Cited by 22
Zou, J., Qin, Y., & Rhee, M. (2024) · Innovation and technology studies. · doi:10.61784/its4026 · Cited by 33
Sakai, Y., Nohejl, A., Hang, J., Kamigaito, H., & Watanabe, T. (2024) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2024.blackboxnlp-1.31 · Cited by 20
Nazar, W., Nazar, G., Kamińska, A., & Daniłowicz‐Szymanowicz, L. (2025) · Journal of Medical Internet Research · doi:10.2196/70481 · Cited by 20
Navigating the Landscape of Hint Generation Research: From the Past to the Future
Jangra, A., Mozafari, J., Jatowt, A., & Muresan, S. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00751 · Cited by 20
AcuGPT-Agent: An LLM-powered intelligent system for acupuncture-based infertility treatment
Wen, J., Liu, D., Xie, Y., Ren, Y., Wang, J., Xia, Y., & Zhu, P. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131116 · Cited by 21
A survey of large language models for data challenges in graphs
Li, M., Zhang, P., Xing, W., Zheng, Y., Zaporojets, K., Chen, J., Zhang, R., Zhang, Y., Gong, S., Hu, J. S., et al. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129643 · Cited by 20
Self-citationRJUA-QA: A Comprehensive QA Dataset for UrologyPreprint
Lyu, S., Chi, C., Cai, H., Shi, L., Xiaoyan, Y., Liu, L., Chen, X., Zhao, D., Zhang, Z., Lyu, X., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.09785 · Cited by 20
Context-aware Decoding Reduces Hallucination in Query-focused SummarizationPreprint
Xu, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.14335 · Cited by 20
Kim, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.03004 · Cited by 20
Han, X., Gao, X. S., Qu, X., & Yu, Z. C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.14321 · Cited by 20
Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
Gallifant, J., Chen, S., Moreira, P., Munch, N., Gao, M., Pond, J., Celi, L. A., Aerts, H., Hartvigsen, T., & Bitterman, D. (2024) · arXiv · doi:10.48550/arxiv.2406.12066 · Cited by 19
MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning
Li, S., Balachandran, V., Feng, S., Ilgen, J., Pierson, E., Koh, P., & Tsvetkov, Y. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-0908 · Cited by 19
“As shown in Figure 1, standard medical question-answering (QA) tasks are formulated in a single-turn setup where all necessary information is provided upfront, and the model is not expected to interact with users (Jin et al., 2021; Pal et al., 2022; Jin et al., 2019; Hendrycks et al., 2020).”
Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
Gallifant, J., Chen, S., Moreira, P. J. F., Munch, N., Gao, M., Pond, J., Celi, L. A., Aerts, H., Hartvigsen, T., & Bitterman, D. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.726 · Cited by 19
Cheng, P., Zhang, K., Lyu, M., Liu, H., Sun, L., & Wu, Y. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104946 · Cited by 19
M o N a C o : More Natural and Complex Questions for Reasoning Across Dozens of Documents
Wolfson, T., Trivedi, H. K., Geva, M., Goldberg, Y., Roth, D., Khot, T., Sabharwal, A., & Tsarfaty, R. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.64 · Cited by 20
Kulshreshtha, D., Belfer, R., Serban, I. V., & Reddy, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.566 · Cited by 19
Şensoy, E., & Çıtırık, M. (2023) · International Ophthalmology · doi:10.1007/s10792-023-02893-x · Cited by 19
ShennongMGS: An LLM-based Chinese Medication Guidance System
Dou, Y., Huang, Y., Zhao, X., Zou, H., Shang, J., Lu, Y., Yang, X., Xiao, J., & Peng, S. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3658451 · Cited by 19
KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server
Wang, W., Liang, X., Ye, R., Chai, J., Chen, S., & Wang, Y. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.438 · Cited by 19
Learning Contextualized Document Representations for Healthcare Answer Retrieval
Arnold, S., van Aken, B., Grundmann, P., Gers, F. A., & Löser, A. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380208 · Cited by 19
Hierarchical Sliding Inference Generator for Question-driven Abstractive Answer Summarization
Li, B., Yang, P., Zhao, H., Zhang, P., & Liu, Z. (2022) · ACM Trans. Inf. Syst. · doi:10.1145/3511891 · Cited by 17
Self-citationExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination
Li, D., Yu, J., Hu, B., Xu, Z., & Zhang, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.129 · Cited by 19
LLMMaps -- A Visual Metaphor for Stratified Evaluation of Large Language ModelsPreprint
Puchert, P., Poonam, P., van Onzenoodt, C., & Ropinski, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.00457 · Cited by 19
Yang, Y., Sun, H., Li, J., Liu, R., Li, Y., Liu, Y., Huang, H., & Gao, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.15777 · Cited by 20
Belyi, M., Friel, R., Shao, S., & Sanyal, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.00975 · Cited by 21
EnronQA: Towards Personalized RAG over Private DocumentsPreprint
Ryan, M. J., Xu, D., Nivera, C., & Campos, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.00263 · Cited by 20
Rethinking Reward Supervision: Rubric-Conditioned Self-DistillationPreprint
Gu, S., Chen, J., Zhou, S., Cohan, A., & Ying, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.19327 · Cited by 19
Improving the robustness and accuracy of biomedical language models through adversarial training
Moradi, M., & Samwald, M. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104114 · Cited by 18
Wang, D., Ye, J., Li, J., Liang, J., Zhang, Q., Hu, Q., Pan, C., Wang, D., Liu, Z. Y., Shi, W. H., et al. (2025) · JMIR Medical Education · doi:10.2196/70190 · Cited by 18
Self-citationCorbeil, J.-P., Abacha, A. B., Michalopoulos, G., Swazinna, P., Del-Agua, M., Tremblay, J., Daniel, A. J., Bader, C., Cho, K., Krishnan, P., et al. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-industry.58 · Cited by 18
Ding, H., Zhao, Y., Hu, T., Patwardhan, M. S., & Cohan, A. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.303 · Cited by 18
Supplementing domain knowledge to BERT with semi-structured information of documents
Chen, J., Wei, Z., Wang, J., Wang, R., Gong, C., Zhang, H., & Miao, D. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121054 · Cited by 18
Balde, G., Roy, S., Mondal, M., & Ganguly, N. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/683 · Cited by 18
MindLLM: Lightweight large language model pre-training, evaluation and domain application
Yang, Y., Sun, H., Li, J., Liu, R., Li, Y., Liu, Y., Gao, Y., & Huang, H. (2024) · AI Open · doi:10.1016/j.aiopen.2024.08.001 · Cited by 18
ChemLit-QA: a human evaluated dataset for chemistry RAG tasks
Wellawatte, G. P., Guo, H., Lederbauer, M., Borisova, A., Hart, M., Brucka, M., & Schwaller, P. (2025) · Machine Learning Science and Technology · doi:10.1088/2632-2153/adc2d6 · Cited by 18
Medical Implications of LLM Based Clinical Decision Support Systems in Healthcare
Ogdu, C. U., Gürbüz, S., Karaköse, M., & Hanoğlu, E. (2025) · doi:10.1109/it64745.2025.10930252 · Cited by 18
Reviewing clinical knowledge in medical large language models: Training and beyond
Li, Q. Y., Liu, H., Guo, C., Gao, C., Chen, D., Wang, M., Gao, F., van Harmelen, F., & Gu, J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114215 · Cited by 20
Kim, H., Lee, S., You, S. C., Huh, S., Kim, J.-E., Kim, S.-T., Ko, D.-R., Kim, J. H., Lee, J. H., Lim, J. S., et al. (2025) · JMIR Medical Informatics · doi:10.2196/76848 · Cited by 20
PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations
Li, J., Hu, R., Huang, K., Zhuang, Y., Liu, Q., Zhu, M., Shi, X., & Wei, L. (2024) · neural information processing systems · doi:10.52202/079017-0341 · Cited by 19
Improving Deep Learning Optimization through Constrained Parameter Regularization
Franke, J. K. H., Hefenbrock, M., Koehler, G., & Hutter, F. (2024) · neural information processing systems · doi:10.52202/079017-0286 · Cited by 18
Experience Retrieval-Augmentation with Electronic Health Records Enables Accurate Discharge QA
Ou, J., Huang, T., Zhao, Y., Yu, Z., Lu, P., Shen, Y., & Ying, R. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1073 · Cited by 18
Deng, Y., Zhang, W.-X., Li, Y.-L., Yang, M., Lam, W., & Shen, Y. (2020) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401208 · Cited by 18
DATLMedQA: A Data Augmentation and Transfer Learning Based Solution for Medical Question Answering
Zhou, S., & Zhang, Y. (2021) · Applied Sciences · doi:10.3390/app112311251 · Cited by 19
Biomedical Question Answering: A Survey of Methods and Datasets
Kaddari, Z., Mellah, Y., Berrich, J., Bouchentouf, T., & Belkasmi, M. G. (2020) · International Conference on Intelligent Computing in Data Sciences (ICDS) · doi:10.1109/icds50568.2020.9268742 · Cited by 18
Minimizing Factual Inconsistency and Hallucination in Large Language ModelsPreprint
Muneeswaran, I., Saxena, S., Prasad, S., Prakash, M., Shankar, A., Varun, V., Vaddina, V., & Gopalakrishnan, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.13878 · Cited by 18
Improving Domain Adaptation through Extended-Text Reading ComprehensionPreprint
Jiang, T., Huang, S., Luo, S., Zhang, Z., Huang, H., Wei, F., Deng, W., Sun, F., Zhang, Q., Wang, D., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.07284 · Cited by 18
Domain-Specific Pretraining of Language Models: A Comparative Study in the Medical FieldPreprint
Kerner, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.14076 · Cited by 18
LLM360 K2: Building a 65B 360-Open-Source Large Language Model from ScratchPreprint
Liu, Z., Tan, B., Wang, H., Neiswanger, W., Tao, T., Li, H., Koto, F., Wang, Y., Sun, S., Pangarkar, O., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.07124 · Cited by 18
Self-citationF2LLM Technical Report: Matching SOTA Embedding Performance with 6 Million Open-Source DataPreprint
Zhang, Z., Liao, Z.-H., Yu, H., Di, P., & Wang, R. (2025) · arXiv.org · doi:10.48550/arxiv.2510.02294 · Cited by 8
Self-Adaptive Cognitive Debiasing for Large Language Models in Decision-MakingPreprint
Lyu, Y., ren, S. D., Feng, Y., Wang, Z., Chen, Z., Ren, Z., & de Rijke, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.04141 · Cited by 18
Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We NeedPreprint
Dedhia, B., Kansal, Y., & Jha, N. K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.13966 · Cited by 18
Diversity Boosts AI-Generated Text Detection
Basani, A. R., & Chen, P. (2026) · Transactions on machine learning research · doi:10.48550/arxiv.2509.18880 · Cited by 19
Shi, H., Zheng, T., Wang, W., Xu, B., Li, C., Chan, C., Fan, T., Song, Y., & Yang, Q. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.16303 · Cited by 18
Beyond Accuracy: Investigating Error Types in GPT-4 Responses to USMLE Questions
Roy, S., Khatua, A., Ghoochani, F., Hadler, U., Nejdl, W., & Ganguly, N. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657882 · Cited by 17
Auditing medical multi-agent AI reveals risks of false consensus
Zhu, Y., Gu, L., Wang, Z., Sang, H., Sui, D., Tang, W., Mi, L., Wang, Y., Gao, J., Yao, L., et al. (2025) · arXiv · doi:10.48550/arxiv.2510.10185 · Cited by 17
Towards Mitigating Hallucination in Large Language Models via Self-Reflection
Ji, Z., Yu, T., Xu, Y., Lee, N., Ishii, E., & Fung, P. (2023) · arXiv · doi:10.48550/arxiv.2310.06271 · Cited by 17
“These systems respond to various question formats, including Yes/No (Tsatsaronis et al., 2015; Jin et al., 2019), multi-choice (Pal et al.”
BackgroundInfluentialBigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
Fries, J. A., Weber, L., Seelam, N., Altay, G., Datta, D., Garda, S., Kang, M., Su, R., Kusa, W., Cahyawijaya, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.15076 · Cited by 17
A survey on large language models in biology and chemistry
Ashyrmamatov, I., Gwak, S. J., Jin, S.-Y., Jun, I., Ucak, U. V., Lee, J. Y., & Lee, J. (2026) · Experimental & Molecular Medicine · doi:10.1038/s12276-025-01583-1 · Cited by 17
PaperTrail: A Claim-Evidence Interface for Grounding Provenance in LLM-based Scholarly Q&A
Martin, A., Leckey, C. A. C., Brown, M. C., & Kaur, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791101 · Cited by 18
Civettini, I., Zappaterra, A., Granelli, B. M., Rindone, G., Aroldi, A., Bonfanti, S., Colombo, F., Fedele, M., Grillo, G., Parma, M., et al. (2023) · British Journal of Haematology · doi:10.1111/bjh.19200 · Cited by 17
MedREQAL: Examining Medical Knowledge Recall of Large Language Models via Question Answering
Vladika, J., Schneider, P., & Matthes, F. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.860 · Cited by 17
Explanatory argument extraction of correct answers in resident medical exams
Goenaga, I., Atutxa, A., Gojenola, K., Oronoz, M., & Agerri, R. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102985 · Cited by 18
MAP: evaluation and multi-agent enhancement of large language models for inpatient pathways
Chen, Z., Peng, Z., Liang, X., Wang, C., Liang, P., Zeng, L., Ju, M., & Yuan, Y. (2026) · npj Health Systems · doi:10.1038/s44401-026-00085-0 · Cited by 18
quEHRy: a question answering system to query electronic health records
Soni, S., Datta, S., & Roberts, K. E. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad050 · Cited by 17
List-wise learning to rank biomedical question-answer pairs with deep ranking recursive autoencoders
Yan, Y., Zhang, B., Li, X., & Liu, Z. (2020) · PLoS ONE · doi:10.1371/journal.pone.0242061 · Cited by 17
Self-citationMeDiaQA: A Question Answering Dataset on Medical DialoguesPreprint
Suri, H., Zhang, Q., Huo, W., Liu, Y., & Guan, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.08074 · Cited by 17
BioTABQA: Instruction Learning for Biomedical Table Question AnsweringPreprint
Luo, M., Saxena, S., Mishra, S. R., Parmar, M., & Baral, C. R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.02419 · Cited by 17
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive BenchmarkPreprint
Liu, F., Li, Z., Zhou, H., Yin, Q., Yang, J., Tang, X., Luo, C., Zeng, M. Y., Jiang, H., Gao, Y., et al. (2024) · medRxiv · doi:10.1101/2024.04.24.24306315 · Cited by 17
Metadata
- Authors
- Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X.
- Affiliations
- University of Pittsburgh (US)
- Carnegie Mellon University (US)
- Google (United States) (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 2567-2577
- OpenAlex
- W2970482702
- Cited by
- 685
- References
- 26
- Citations by year
- 2026: 188 · 2025: 215 · 2024: 107 · 2023: 75 · 2022: 36 · 2021: 39 · 2020: 25
- FWCI
- 75.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- question answering, biomedical research, PubMed, scientific publications, datasets, biomedical question answering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.