The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
Open accessAbstract
, to evaluate performance differences on medical benchmarks after swapping brand and generic drug names using physician expert annotations. We assess both open-source and API-based LLMs on MedQA and MedMCQA, revealing a consistent performance drop ranging from 1-10%. Furthermore, we identify a potential source of this fragility as the contamination of test data in widely used pre-training datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs can accurately match brand and generic drug names.
“Because we previously showed that LLMs can accurately match brand and generic drug names30, this allowed for a controlled and scalable experimental setup to characterize LLM sycophantic compliance to illogical requests.”
From When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior (2025) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 13
- Arefyev Nikolay, Sheludko Boris, Podolskiy Alexander, and Panchenko Alexander. 2020. Always keep your target in mind: Studying semantics and improving performance of neural lexical substitution. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1242– 1255, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- Brown Tom, Mann Benjamin, Ryder Nick, Subbiah Melanie, Jared D Kaplan Prafulla Dhariwal, Neelakantan Arvind, Shyam Pranav, Sastry Girish, Askell Amanda, 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877– 1901.
- Han Tessa, Kumar Aounon, Agarwal Chirag, and Lakkaraju Himabindu. 2024. Towards safe large language models for medicine. Preprint, arXiv:2403.03744. · arXiv
- Jin Di, Pan Eileen, Oufattole Nassim, Weng Wei-Hung, Fang Hanyi, and Szolovits Peter. 2020. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. arXiv preprint arXiv:2009.13081. · arXiv
- Jin Qiao, Dhingra Bhuwan, Liu Zhengping, Cohen William, and Lu Xinghua. 2019. PubMedQA: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2567– 2577, Hong Kong, China. Association for Computational Linguistics.
- Liu Junling, Zhou Peilin, Hua Yining, Chong Dading, Tian Zhongyu, Liu Andrew, Wang Helin, You Chenyu, Guo Zhenhua, Lei Zhu, and Li Michael Lingzhi 2023. Benchmarking large language models on CMExam - a comprehensive chinese medical exam dataset. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
- Raffel Colin, Shazeer Noam, Roberts Adam, Lee Katherine, Narang Sharan, Matena Michael, Zhou Yanqi, Li Wei, and Liu Peter J. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv e-prints.
- Recht Benjamin, Roelofs Rebecca, Schmidt Ludwig, and Shankar Vaishaal. 2019. Do imagenet classifiers generalize to imagenet? In International conference on machine learning, pages 5389– 5400. PMLR.
- Riedl Martin, Glass Michael, and Gliozzo Alfio. 2014. Lexical substitution for the medical domain. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 610– 614, Doha, Qatar. Association for Computational Linguistics.
- Soldaini Luca, Kinney Rodney, Bhagia Akshita, Schwenk Dustin, Atkinson David, Authur Russell, Bogin Ben, Chandu Khyathi, Dumas Jennifer, Elazar Yanai, Hofmann Valentin, Ananya Harsh Jha Sachin Kumar, Lucy Li, Lyu Xinxi, Lambert Nathan, Magnusson Ian, Morrison Jacob, Muennighoff Niklas, Naik Aakanksha, Nam Crystal, Peters Matthew E., Ravichander Abhilasha, Richardson Kyle, Shen Zejiang, Strubell Emma, Subramani Nishant, Tafjord Oyvind, Walsh Pete, Zettlemoyer Luke, Smith Noah A., Hajishirzi Hannaneh, Beltagy Iz, Groeneveld Dirk, Dodge Jesse, and Lo Kyle. 2024. Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research. arXiv preprint.
- Wen Zhi, Lu Xing Han, and Reddy Siva. 2020. MeDAL: Medical abbreviation disambiguation dataset for natural language understanding pretraining. In Proceedings of the 3rd Clinical Natural Language Processing Workshop, pages 130– 135, Online. Association for Computational Linguistics.
- Yuret Deniz. 2007. KU: Word sense disambiguation by substitution. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 207– 214, Prague, Czech Republic. Association for Computational Linguistics.
- Zhou Wangchunshu, Ge Tao, Xu Ke, Wei Furu, and Zhou Ming. 2019. BERT-based lexical substitution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3368– 3373, Florence, Italy. Association for Computational Linguistics.
Cited by
Sorted by citations
The TRIPOD-LLM reporting guideline for studies using large language models
Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G., Demner-Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03425-5 · Cited by 615
Self-citationChen, S., Gao, M., Sasse, K., Hartvigsen, T., Anthony, B., Fan, L., Aerts, H., Gallifant, J., & Bitterman, D. S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02008-z · Cited by 112
Self-citationThe TRIPOD-LLM Statement: A Targeted Guideline For Reporting Large Language Models UsePreprint
Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G. E., Demner‐Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C. O., et al. (2024) · medRxiv · doi:10.1101/2024.07.24.24310930 · Cited by 21
Self-citationProCyon: A multimodal foundation model for protein phenotypesPreprint
Queen, O., Huang, Y., Calef, R., Giunchiglia, V., Chen, T., Dasoulas, G., Tai, L., Abbadessa, G., Howell, O., Li, M. M., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.10.627665 · Cited by 14
Self-citationde Matos, J. C. R. G., Gallifant, J., Pei, J., & Wong, A.-K. I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82007-6_20 · Cited by 8
Self-citationKnauer, R., Koddenbrock, M., Wallsberger, R., Brisson, N. M., Duda, G. N., Falla, D., Evans, D. W., & Rodner, E. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736818 · Cited by 9
Zhao, X., Blotske, K., Cargile, M., Tilley, A., Murray, B., Gao, Y., Henry, K., Smith, S. E., Barreto, E. F., Bauer, S. R., et al. (2025) · medRxiv · doi:10.64898/2025.12.01.25341004 · Cited by 5
A machine learning approach for automating review of a RxNorm medication mapping pipeline output
Hüser, M., Doole, J., Pinho, V., Rouhizadeh, H., Teodoro, D., Saiyed, A., & Palchuk, M. B. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104909 · Cited by 3
Reliability of Large Language Model Knowledge Across Brand and Generic Cancer Drug Names
Gallifant, J., Chen, S., Jain, S. K., Moreira, P. A. M., Topaloĝlu, Ü., Aerts, H. J. W. L., Warner, J. L., Cava, W. L., & Bitterman, D. S. (2025) · JCO Clinical Cancer Informatics · doi:10.1200/cci-24-00257 · Cited by 2
Self-citationSun, X., Hong, Q., Zhang, M., Li, Y., Chen, T., Huang, Z., Liang, G., Tang, W., Xu, S., Ni, X., et al. (2026) · Cell Reports Medicine · doi:10.1016/j.xcrm.2025.102547 · Cited by 3
Zheng, Y., Whitaker, M., Geer, A., Gong, J., & Lester, C. A. (2025) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi64645.2025.00020 · Cited by 1
Drug or Pokémon? Large language model performance in identification of fabricated medicationsPreprint
Henry, K., Murray, B., Zhao, X., Blotske, K., Gao, Y., Smith, B. A., Smith, S. E., Barreto, E. F., Bauer, S. R., Sohn, S., et al. (2026) · medRxiv · doi:10.64898/2026.01.12.26343930 · Cited by 1
Self-citationReasoning Over Pre-training: Evaluating LLM Performance and Augmentation in Women’s HealthPreprint
Imprialou, M., Kaltsas, N., Oliinyk, V., Vigrass, T., Schwarzmann, J., Rosenthal, R., Glastonbury, C. A., Wigley, C., Gillam, M., Kanani, N., et al. (2025) · medRxiv · doi:10.1101/2025.05.22.25328162
AI-Powered Drug-Substance Prediction: A Comparative Study of ML, LM, and LLM in Clinical Pharmacy
Couto, J., Tabbal, A., Borges, O. T., Engelmann, D. C., dos Santos, H. D. P., Arocha, M., Oliveira, J. D., & dos Santos, C. H. M. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-07109-5_2
O’Connell, C. P., Adibi, A., Dozie-Nnamah, P., & Alsentzer, E. (2026) · medRxiv · doi:10.64898/2026.02.12.26346164 · Cited by 1
How Well Does ChatGPT Understand Drug Mechanisms? A Knowledge + Reasoning Evaluation Dataset
Mohan, S., & Karaletsos, T. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11391095
IndiHealthBench: Evaluating LLMs for Clinical Translation Across Indian Linguistic Diversity
Singh, P., Saraf, J., Sharma, A., Ujjain, S., Jain, P., & Kumar, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-18480-1_59
Safety and Reliability Evaluation of Medical Chatbots Powered by Large Language Models
Nurfadhilah, E., Aini, L. R., Santosa, A., Uliniansyah, M. T., Gunarso, G., Putra, P. K., Pebiana, S., Fajri, R., Hidayati, N. N., Prastowo, R. D., et al. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7862-6.ch003
Robustness Gap of Large Language Models in NephrologyPreprint
Soejima, A., Kitano, F., Ichikawa, D., Shibagaki, Y., & Noda, R. (2026) · medRxiv · doi:10.64898/2026.08.17.26360565
Hazra, M., & Anandaram, H. (2026) · Frontiers in Pharmacology · doi:10.3389/fphar.2026.1915973
Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation
Caruzzo, C., Yoo, D., & Kim, T. S. (2026) · arXiv · doi:10.48550/arxiv.2607.09349
Metadata
- Authors
- Gallifant, J., Chen, S., Moreira, P. J. F., Munch, N., Gao, M., Pond, J., Celi, L. A., Aerts, H., Hartvigsen, T., & Bitterman, D.
- Affiliations
- Massachusetts Institute of Technology (US)
- Boston Children's Hospital (US)
- Universitat Pompeu Fabra (ES)
- Aarhus University (DK)
- Harvard University (US)
- Beth Israel Deaconess Medical Center (US)
- University of Virginia (US)
- Corresponding author
- Bitterman, Danielle
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2024
- Published
- 2024
- Volume
- 2024
- Pages
- 12448-12465
- PMC
- PMC12671478
- OpenAlex
- W4404781405
- Cited by
- 19
- References
- 0
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 2
- FWCI
- 5.61
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2148451)
- European Commission
- National Institutes of Health (OT2OD032701, Bridge2AI, 5r01eb017205-07, R35CA22052, R01-EB017205)
- National Cancer Institute (U01CA190234, U24CA194354, U54CA274516, R01CA294033, U01CA209414)
- Fogarty International Center (U54TW012043)
- National Institute of Biomedical Imaging and Bioengineering (R01-EB017205)
- H2020 European Research Council (866504)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Keywords
- drug names, MedQA, MedMCQA, medical question answering, pre-training dataset, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
