PubMedQA: A Dataset for Biomedical Research Question Answering
Open accessThe DOI matches its Crossref record; 24 of the 26 references it cites were found in a registry or an index.
Abstract
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, Xinghua Lu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“given that medical question datasets such as PubMedQA42 and MedQA43 consist of fixed question sets susceptible to memorization”
From An automated framework for assessing how well LLMs cite relevant medical references (2025) · cites this paper as [43]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“All analyses were performed on publicly available, de-identified benchmark datasets—MIRAGE [23], PubMedQA [24], and the medical licensing examination questions of Katz et al. [25]—used in accordance with their respective licenses and terms of use.”
From The economics of accuracy for medical reasoning with large language models (2026) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“PubMedQA (Jin et al., 2019) is a biomedical research QA dataset under the MIT license. It contains 1,000 expert-annotated questions ( pqa\_labeled\_split ) and 211K machine-labeled questions from PubMed abstracts (the most widely used biomedical literature resource).”
From MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025) · cites this paper as Jin et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Notably, in healthcare, medical NLP benchmarks like MedQA7, PubMedQA8 and the Massive Multitask Language Understanding (MMLU) serve as the de-facto reporting standard for state-of-the-art medical LLMs9–11 with claims of ‘superhuman’ performance in patient-facing tasks12.”
From Medical large language models are vulnerable to data-poisoning attacks (2025) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“The ground truth question-answer pairs are sourced from the pqa\_labeled split of PubMedQA (Jin et al., 2019), which contains 1,000 expert-annotated samples, supplemented with 9,000 instances randomly selected from the pqa\_artificial split.”
From MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025) · cites this paper as Jin et al., 2019
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 2×
“We fine-tune BioBERT (Lee et al., 2019) on Pub-MedQA as a baseline.”
MethodThe Stanford CoreNLP Natural Language Processing Toolkit
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014) · Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.3115/v1/p14-5010 · Cited by 7,037
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“Unlike other QA datasets in which questions are asked by crowd-workers for existing contexts (Rajpurkar et al., 2016; Yang et al., 2018; Kočiskỳ et al., 2018), in PubMedQA contexts are generated to answer the questions and both are written by the same authors.”
BackgroundNatural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 6×
“, 2018), Natural Questions (Kwiatkowski et al., 2019), ShARC (Saeidi et al.”
BackgroundMethodInfluentialBoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1300 · Cited by 1,855
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 6×
“Supporting fact for the answer is highlighted . et al., 2018; Yang et al., 2018; Kwiatkowski et al., 2019), the largest annotated biomedical QA dataset, BioASQ (Tsatsaronis et al., 2015) has less than 3k training instances, most of which are simple factual questions.”
BackgroundMethodInfluentialSelf-citationEnhanced LSTM for Natural Language Inference
Chen, Q., Zhu, X., Ling, Z.-H., Wei, S., Jiang, H., & Inkpen, D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1152 · Cited by 1,113
Research works printed but not linked: 1
- 2012 Conference and Labs of the Evaluation ForumQuestion Answering For Machine Reading Evaluation (QA4MRE), Rome/Forner, J.[edit.]; ea , pages 1-14.
Cited by
Items 501–600 of 1,026
Reward-RAG: Enhancing RAG with Reward Driven SupervisionPreprint
Nguyen, T., Chin, P. S., & Tai, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.03780 · Cited by 17
Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass
Chen, T., Fang, H., Xia, P., Liu, X., Durme, B. V., Zettlemoyer, L., Gao, J., & Cheng, H. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2411.05877 · Cited by 18
scBench: Evaluating AI Agents on Single-Cell RNA-seq AnalysisPreprint
Workman, K., Yang, Z., Muralidharan, H. S., Abdulali, A., & Le, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.09063 · Cited by 17
Generative Representational Instruction Tuning
Muennighoff, N., Su, H., Wang, L., Yang, N., Wei, F., Yu, T., Singh, A., & Kiela, D. (2024) · arXiv · doi:10.48550/arxiv.2402.09906 · Cited by 16
DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm
Zhu, X., Ren, Y., Fang, F., Tan, Q., Wang, S., & Cao, Y. (2025) · neural information processing systems · doi:10.52202/085713-0065 · Cited by 16
SQuAI: Scientific Question-Answering with Multi-Agent Retrieval-Augmented Generation
Besrour, I., He, J., Schreieder, T., & Färber, M. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761471 · Cited by 17
The Aloe Family recipe for open and specialized healthcare LLMs
García-Gasulla, D., Bayarri-Planas, J., Gururajan, A. K., Lopez-Cuena, E., Tormos, A., Hinjos, D., Bernabeu-Pérez, P., Arias-Duart, A., Martin-Torres, P. A., Mallo, M. G., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02637-y · Cited by 16
Cusumano, G., D'arrigo, S., Terminella, A., & Lococo, F. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13133750 · Cited by 16
PubMed Retrieval with RAG Techniques
Thomo, A. (2024) · Studies in health technology and informatics · doi:10.3233/shti240498 · Cited by 16
Kadhim, A. K., Jiao, L., Shafik, R., & Granmo, O. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-31141-2_10 · Cited by 16
An Empirical Evaluation of Large Language Models on Consumer Health Questions
Abrar, M., Sermet, Y., & Demir, İ. (2025) · BioMedInformatics · doi:10.3390/biomedinformatics5010012 · Cited by 17
Medication information extraction using local large language models
Richter-Pechanski, P., Seiferling, M., Kiriakou, C., Schwab, D. M., Geis, N. A., Dieterich, C., & Frank, A. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104898 · Cited by 16
A survey on large language models unlearning: taxonomy, evaluations, and future directions
Le-Khac, U. N., & Truong, V. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11376-7 · Cited by 16
Advancing Problem-Based Learning in Biomedical Engineering in the Era of Generative AI
Nnamdi, M. C., Tamo, J. B., Marteau, B. L., Shi, W., & Wang, M. D. (2026) · IEEE Transactions on Education · doi:10.1109/te.2026.3658007 · Cited by 16
Self-Routing RAG: Binding Selective Retrieval with Knowledge Verbalization
Wu, D., Gu, J.-C., Chang, K.-W., & Peng, N. (2026) · doi:10.18653/2026.findings-aacl.28 · Cited by 16
Tutorial: guidance on the use of large language models for medical research
Jin, Q., Wan, N., Leaman, R., Tian, S., Wang, Z., Yang, Y., Wang, Z., Xiong, G., Lai, P.-T., Zhu, Q. L., et al. (2026) · Nature Protocols · doi:10.1038/s41596-026-01408-z · Cited by 16
Self-citationBioMamba: Domain-Adaptive Biomedical Language Models
Yue, L., Zhu, M., Xing, S., Cao, Y., Wang, Y., Chenthamarakshan, V., Pan, S., Das, P., & Fu, T. (2026) · Health Data Science · doi:10.34133/hds.0454 · Cited by 16
An Empirical Exploration in Quality Filtering of Text DataPreprint
Gao, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.00698 · Cited by 16
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
Chen, Y., Wang, S., Zhang, Y., Lin, Z., Zhang, H., Sun, W., Ding, T., & Sun, R. (2025) · Transactions on machine learning research · doi:10.48550/arxiv.2407.20999 · Cited by 17
MEG: Medical Knowledge-Augmented Large Language Models for Question AnsweringPreprint
Cabello, L., Martin-Turrero, C., Akujuobi, U., Søgaard, A., & Bobed, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.03883 · Cited by 16
Reinventing Clinical Dialogue: Agentic Paradigms for LLM Enabled Healthcare CommunicationPreprint
Zhi, X., Zhao, H., Wu, L., Zhao, C., & Zhu, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.01453 · Cited by 16
Text summarization via global structure awareness
Zhang, J., Zhang, C., Chen, S., Liu, Y., Li, C., Sun, Q., Yuan, S., Puspitasari, F. D., Han, D., Wang, G., et al. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2602.09821 · Cited by 16
Jev in Medicine: A Benchmark EvaluationPreprint
Madrid-García, A., & Barbancho, B. M. (2026) · arXiv (Cornell University) · doi:10.5281/zenodo.23008155 · Cited by 16
DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domains
Chen, Z., He, K., Huang, Y., Zhu, Y., & Feng, M. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.971 · Cited by 15
Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs
Kadhe, S., Ahmed, F., Wei, D., Baracaldo, N., & Padhi, I. (2026) · Synthesis lectures on computer vision · doi:10.1007/978-3-032-17282-2_7 · Cited by 15
Lee, D., Rim, D., Choi, M., & Choo, J. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.936 · Cited by 15
Single and Multi-Hop Question-Answering Datasets for Reticular Chemistry with GPT-4-Turbo
Rampal, N., Wang, K., Burigana, M., Hou, L., Al-Johani, J., Sackmann, A., Murayshid, H. S., Al-Sumari, W. A., Al-Abdulkarim, A. M., Alhazmi, N. E., et al. (2024) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.4c00805 · Cited by 15
PodGPT: an audio-augmented large language model for research and education
Jia, S., Bit, S., Searls, E., Lauber, M. V., Fan, P., Wang, W. M., Claus, L. A., Jasodanand, V. H., Veerapaneni, D., Au, R., et al. (2025) · npj Biomedical Innovations. · doi:10.1038/s44385-025-00022-0 · Cited by 15
Fu, J., Guo, C., & Li, C. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754862 · Cited by 17
CUB: Benchmarking Context Utilisation Techniques for Language Models
Hagström, L., Kim, Y., Yu, H., Lee, S.-G., Johansson, R., Cho, H., & Augenstein, I. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1151 · Cited by 15
Bianchi, O., Willey, M., Alvarado, C. X., Danek, B. P., Khani, M., Kuznetsov, N., Dadu, A., Shah, S. M. T., Koretsky, M. J., Makarious, M. B., et al. (2026) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100943 · Cited by 17
Testoni, A., & Calixto, I. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.106 · Cited by 15
SentiMedQAer: A Transfer Learning-Based Sentiment-Aware Model for Biomedical Question Answering
Zhu, X., Chen, Y., Gu, Y., & Xiao, Z. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.773329 · Cited by 15
Enriching Biomedical Knowledge for Low-resource Language Through Large-scale Translation
Phan, L., Dang, T., Tran, H., Trinh, T. H., Phan, V., Chau, L. D., & Luong, M.-T. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.228 · Cited by 15
A Statistical Turing Test for Generative ModelsPreprint
Helm, H. S., Priebe, C. E., & Yang, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.08913 · Cited by 15
Park, J., Bae, J., Lee, U., Ahn, T., Lee, S., Kim, D.-H., Choi, A., Jeong, Y., Moon, J., & Kim, H. (2024) · arXiv (Cornell University) · doi:10.13140/rg.2.2.13490.82883 · Cited by 15
Aligning (Medical) LLMs for (Counterfactual) FairnessPreprint
Poulain, R., Fayyaz, H., & Beheshti, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.12055 · Cited by 15
No Mean Feat: Simple, Strong Baselines for Context CompressionPreprint
Feldman, Y., & Artzi, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.20797 · Cited by 15
MedEthicsQA: A Comprehensive Question Answering Benchmark for Medical Ethics Evaluation of LLMsPreprint
Wei, J., Meng, Z., Xiao, Z., Hu, T., Feng, Y., Zhou, Z., Wu, J., & Liu, Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.22808 · Cited by 15
Self-citationLi, C., Elmahdy, A., Boyd, A., Wang, Z., Zeng, S., Garcia, A., Bhatia, P., Kass‐Hout, T., Xiao, C., & Hong, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.20718 · Cited by 15
Min, D., Zhang, K., Wu, T., & Cheng, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.19134 · Cited by 15
EnterpriseRAG-Bench: A RAG Benchmark for Company Internal KnowledgePreprint
Sun, Y., Rahmfeld, J., Weaver, C., Chen, W., Desai, R., Huang, W., & Butler, M. H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.05253 · Cited by 15
Cusumano, G., D'arrigo, S., Terminella, A., & Lococo, F. (2024) · doi:10.20944/preprints202405.0759.v1 · Cited by 14
MultifacetEval: Multifaceted Evaluation to Probe LLMs in Mastering Medical Knowledge
Zhou, Y., Liu, X., Ning, C., & Wu, J. Y. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/737 · Cited by 14
Balde, G., Roy, S., Mondal, M., & Ganguly, N. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1179 · Cited by 14
Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs
Le, C., Gong, Z., Wang, C., Ni, H., Li, P., & Chen, X. (2025) · International Conference on Artificial Intelligence, Human-Computer Interaction and Natural Language Processing (ICAHN) · doi:10.1109/icahn67688.2025.00016 · Cited by 14
HealthContradict: Evaluating biomedical knowledge conflicts in language models
Zhang, B., Bornet, A., Yang, R., Liu, N., & Teodoro, D. (2026) · npj Digital Medicine · doi:10.1038/s41746-025-02336-0 · Cited by 14
Dyport: dynamic importance-based biomedical hypothesis generation benchmarking technique
Tyagin, I., & Safro, I. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05812-8 · Cited by 14
Biomedical Entity Representation with Graph-Augmented Multi-Objective Transformer
Sakhovskiy, A., Semenova, N., Kadurin, A., & Tutubalina, E. V. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.288 · Cited by 16
Self-Specialization: Uncovering Latent Expertise within Large Language Models
Kang, J., Luo, H., Zhu, Y., Hansen, J., Glass, J. R., Cox, D., Ritter, A., Feris, R., & Karlinsky, L. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.157 · Cited by 14
Generating Synthetic Text Data to Evaluate Causal Inference MethodsPreprint
Wood-Doughty, Z., Shpitser, I., & Dredze, M. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.05638 · Cited by 15
ArcGPT: A Large Language Model Tailored for Real-world Archival ApplicationsPreprint
Zhang, S., Hou, J., Peng, S., Li, Z., Hu, Q., & Wang, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.14852 · Cited by 14
Are Expert-Level Language Models Expert-Level Annotators?Preprint
Tseng, Y.-M., Chen, W., Chen, C.-C., & Chen, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.03254 · Cited by 15
Essential-Web v1.0: 24T tokens of organized web dataPreprint
Essential, A. I., :, Hojel, A., Pust, M., Romanski, T., Vanjani, Y., Kapila, R., Parmar, M., Chaluvaraju, A., Tripathy, A. R., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.14111 · Cited by 14
Structured Outputs Enable General-Purpose LLMs to be Medical ExpertsPreprint
Guo, G.-F., Zhang, K., Hooi, B., Cai, Y.-J., Lu, X.-Q., Peng, N.-Y., & Wang, Y.-W. (2025) · arXiv.org · doi:10.48550/arxiv.2503.03194 · Cited by 12
Self-citationHumans and Large Language Models in Clinical Decision Support: A Study with Medical CalculatorsPreprint
Wan, N. C., Jin, Q., Chan, J., Xiong, G., Applebaum, S. S., Gilson, A., McMurry, R., Taylor, R. A., Zhang, A., Chen, Q., et al. (2024) · PubMed Central · doi:10.48550/arxiv.2411.05897 · Cited by 14
Self-citationWang, X., Huey, S. L., Sheng, R., Mehta, S., & Wang, F. (2025) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.70073 · Cited by 13
Pattern Recognition or Medical Knowledge? The Problem with Multiple-Choice Questions in Medicine
Griot, M., Vanderdonckt, J., Yuksel, D., & Hemptinne, C. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.266 · Cited by 13
“…as ChatGPT, have demonstrated significant potential in the medical field, with studies evaluating their performance on tests originally designed for humans, including the United States Medical Licensing Examination (USMLE) (Jin et al., 2020; Pal et al., 2022; Jin et al., 2019; Nori et al., 2023).”
Pattern Recognition or Medical Knowledge? The Problem with Multiple-Choice Questions in Medicine
Griot, M., Vanderdonckt, J., Yuksel, D., & Hemptinne, C. (2024) · arXiv · doi:10.48550/arxiv.2406.02394 · Cited by 13
Subramanian, A., Schlegel, V., Kashyap, A. R., Nguyen, T.-T., Dwivedi, V. P., & WINKLER, S. S. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.238 · Cited by 13
PNCD: Mitigating LLM hallucinations in noisy environments–A medical case study
Qu, J., Liu, J., Liu, X., Chen, M., Li, J., & Wang, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103328 · Cited by 13
DART: An AIGT Detector using AMR of Rephrased Text
Park, H., Kim, B.-J., & Kim, B. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-short.59 · Cited by 13
Wu, J., Wang, J., Liu, Z., Chen, B., Hu, D., Wu, H., & Xia, S. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.294 · Cited by 14
Self-citationBašaragin, B., Ljajić, A., Medvecki, D., Cassano, L., Košprdić, M., & Milošević, N. (2024) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2024.bionlp-1.44 · Cited by 13
Leveraging Unstructured Text Data for Federated Instruction Tuning of Large Language Models
Ye, R., Ge, R., Yuchi, F., Chai, J., Wang, Y., & Chen, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82240-7_9 · Cited by 13
MedQA-SWE - a Clinical Question & Answer Dataset for Swedish
Hertzberg, N., & Lokrantz, A. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4ckcc2pepdd3 · Cited by 13
A Gated Dilated Convolution with Attention Model for Clinical Cloze-Style Reading Comprehension
Wang, B., Zhang, X., Zhou, X., & Li, J. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17041323 · Cited by 13
Probing Pre-Trained Language Models for Disease Knowledge
Alghanmi, I., Espinosa-Anke, L., & Schockaert, S. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.266 · Cited by 13
What Would it Take to get Biomedical QA Systems into Practice?
Kell, G., Marshall, I. J., Wallace, B., & Jaun, A. Y. R. (2021) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/2021.mrqa-1.3 · Cited by 13
SummerTime: Text Summarization Toolkit for Non-experts
Ni, A., Azerbayev, Z., Mutuma, M., Feng, T., Zhang, Y., Tao, Y., Awadallah, A. H., & Radev, D. (2021) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2021.emnlp-demo.37 · Cited by 13
DocLens: Multi-aspect Fine-grained Evaluation for Medical Text GenerationPreprint
Xie, Y., Zhang, S., Cheng, H., Liu, P., Gero, Z., Wong, C., Naumann, T., Poon, H., & Rose, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.09581 · Cited by 12
Uncertainty Estimation of Large Language Models in Medical Question AnsweringPreprint
Wu, J., Yu, Y., & Zhou, H.-Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.08662 · Cited by 13
Improved precision oncology question-answering using agentic LLMPreprint
Das, R., Maheswari, K., Siddiqui, B. S., Arora, N., Paul, A., Nanshi, J., Udbalkar, V., Sarvade, A., Chaturvedi, H., Shvartsman, T., et al. (2024) · medRxiv · doi:10.1101/2024.09.20.24314076 · Cited by 13
SciDFM: A Large Language Model with Mixture-of-Experts for SciencePreprint
Sun, L., Luo, D., Ma, D., Zhao, Z., Chen, B., Shen, Z., Zhu, S., Chen, L., Chen, X., & Yu, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.18412 · Cited by 13
Bao, G., Zhao, Y., He, J., & Zhang, Y. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2412.11506 · Cited by 13
Med-REFL: Medical Reasoning Enhancement via Self-Corrected Fine-grained ReflectionPreprint
Yang, Z., Qian, J., Peng, Z., Zhang, H., Huang, Y.-A., Tan, K., & Huang, Z.-A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.13793 · Cited by 13
Liao, H., Hu, W., Xu, Y., He, S., Jun, Z., & Liu, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.15774 · Cited by 13
Evaluating Large Language Models as Expert AnnotatorsPreprint
Tseng, Y.-M., Chen, W., Chen, C.-C., & Chen, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.07827 · Cited by 14
MHQA: A Diverse, Knowledge Intensive Mental Health Question Answering Challenge for Language ModelsPreprint
Racha, S., Joshi, P., Raman, A., Jangid, N., Sharma, M., Ramakrishnan, G., & Punjabi, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.15418 · Cited by 13
MedAraBench: Large-Scale Arabic Medical Question Answering Dataset and BenchmarkPreprint
Abu-Daoud, M., Kharouf, L., Hajj, O. E., Samad, D. E., Al-Omari, M., Mallat, J., Saleh, K. A., Habash, N., & Shamout, F. E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.01714 · Cited by 13
Kassem, A., Mahmoud, O., & Saad, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.265 · Cited by 12
Feng, Y., Shen, B., Gu, N., Zhao, J., Fu, P., Lin, Z., & Wang, W. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.951 · Cited by 12
Advancing Healthcare with Large Language Models: Techniques and Application
Hu, Z., Peng, Z., Bi, Z., Shen, Q., Liu, Z., Lou, J., & Luo, X. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125540 · Cited by 12
Self-citationMedCoAct: Confidence-Aware Multi-Agent Collaboration for Complete Clinical Decision
Zheng, H., Shi, Z., & Yi, P. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356820 · Cited by 12
Wang, Z., Wang, H., Feng, S., Yang, X., Wang, D., Zhang, Y., Lin, J., Ji, X., & Yang, H. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.904 · Cited by 12
Long-Tailed Question Answering in an Open World
Dai, Y. Q., Lang, H., Zheng, Y., Huang, F., & Li, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.351 · Cited by 12
Zhang, W., Zeng, K., Xinming, Y., Shi, T., & Wang, P. (2023) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3584371.3613008 · Cited by 12
Real-GPT: Efficiently Tailoring LLMs for Informed Decision-Making in the Real Estate Industry
Gloria, B., Melsbach, J. W., Bienert, S., & Schoder, D. (2024) · Journal of Real Estate Portfolio Management · doi:10.1080/10835547.2024.2372748 · Cited by 13
Dolomites : Domain-Specific Long-Form Methodical Tasks
Malaviya, C., Agrawal, P., Ganchev, K., Srinivasan, P., Huot, F., Berant, J., Yatskar, M., Das, D., Lapata, M., & Alberti, C. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00727 · Cited by 13
RAmBLA: A Framework for Evaluating the Reliability of LLMs as Assistants in the Biomedical DomainPreprint
Bolton, W. J., Poyiadzi, R., Morrell, E., Bueno, G. V. B. G., & Goetz, L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.14578 · Cited by 12
IMAS: A Comprehensive Agentic Approach to Rural Healthcare DeliveryPreprint
Gangavarapu, A., & Gangavarapu, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.12868 · Cited by 12
The Limited Impact of Medical Adaptation of Large Language and Vision-Language ModelsPreprint
Jeong, D. P., Mani, P., Garg, S., Lipton, Z. C., & Oberst, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.08870 · Cited by 12
Ozaki, S., Kato, Y., Feng, S., Tomita, M., Hayashi, K., Hashimoto, W., Obara, R., Oyamada, M., Hayashi, K., Kamigaito, H., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.20309 · Cited by 13
CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document IntelligencePreprint
Ma, D., Li, J., Wang, Z., Wang, Y., Kong, J., Zeng, W., Xiao, J., Yang, J., BangRui, X., Wang, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.12882 · Cited by 12
A neuro-symbolic method for understanding free-text medical evidence.
Kang, T., Turfah, A., Kim, J., Perotte, A., & Weng, C. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab077 · Cited by 11
“The state-of-the-art system for PubMedQA, reported in Jin et al 40 —which is also the top performer on their leaderboard—adopts a multiphase fine-tuning of BioBERT on both labeled and unlabeled data collections.”
Ouyang, Z., Qiu, Y., Wang, L., De Melo, G., Zhang, Y., Wang, Y., & He, L. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.480 · Cited by 11
RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems
Friel, R., Belyi, M., & Sanyal, A. (2024) · arXiv · doi:10.48550/arxiv.2407.11005 · Cited by 11
Qin, Z., He, Z., Zhang, Y., Shen, Y., & Li, K. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112178 · Cited by 11
Ozaki, S., Kato, Y., Feng, S., Tomita, M., Hayashi, K., Hashimoto, W., Obara, R., Oyamada, M., Hayashi, K., Kamigaito, H., et al. (2025) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2025.bionlp-1.1 · Cited by 11
Jeong, S., Choi, Y. H., Kim, J. W., & Jang, B. (2026) · IEEE Access · doi:10.1109/access.2026.3716053 · Cited by 11
Metadata
- Authors
- Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X.
- Affiliations
- University of Pittsburgh (US)
- Carnegie Mellon University (US)
- Google (United States) (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 2567-2577
- OpenAlex
- W2970482702
- Cited by
- 685
- References
- 26
- Citations by year
- 2026: 188 · 2025: 215 · 2024: 107 · 2023: 75 · 2022: 36 · 2021: 39 · 2020: 25
- FWCI
- 75.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- question answering, biomedical research, PubMed, scientific publications, datasets, biomedical question answering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.