Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
Open accessThe DOI matches its Crossref record; 61 of the 63 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on general domain corpora, such as newswire and Web. A prevailing assumption is that even domain-specific pretraining can benefit by starting from general-domain language models. In this article, we challenge this assumption by showing that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models. To facilitate this investigation, we compile a comprehensive biomedical NLP benchmark from publicly available datasets. Our experiments show that domain-specific pretraining serves as a solid foundation for a wide range of biomedical NLP tasks, leading to new state-of-the-art results across the board. Further, in conducting a thorough evaluation of modeling choices, both for pretraining and task-specific fine-tuning, we discover that some common practices are unnecessary with BERT models, such as using complex tagging schemes in named entity recognition. To help accelerate research in biomedical NLP, we have released our state-of-the-art pretrained and task-specific models for the community, and created a leaderboard featuring our BLURB benchmark (short for Biomedical Language Understanding & Reasoning Benchmark) at https://aka.ms/BLURB .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These research efforts present promising results in extracting true ADE mentions from drug label text, indicating the suitability of using NLP methods for this task.”
From OnSIDES database: Extracting adverse drug events from drug labels using natural language processing models (2025) · cites this paper as [22]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Healthcare represents a compelling domain for LLM use because the vast majority of actionable patient data is locked within unstructured clinical text, including electronic health records, clinical notes, and biomedical literature [4,5].”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [5]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“These systems use biomedical embeddings (e.g., BioBERT, PubMedBERT [30]), specialized corpora (e.g., PubMed abstracts and clinical notes), and domain-specific knowledge bases (e.g., unified medical language system (UMLS), MIMIC). By customizing retrieval and generation to medical language, they enable reliable performance in tasks such as diagnostic decision support, clinical report summarization,”
From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [30]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“For example, a number of efforts have fine-tuned BERT models to classify drug label text blocks into predefined sections of the label and outperformed baseline models for this task22–23.”
From OnSIDES database: Extracting adverse drug events from drug labels using natural language processing models (2025) · cites this paper as [22]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“LLMs have quickly surpassed the average human score on the United States Medical Licensing Exam (USMLE) in a short amount of time, from 38.1% in September 20212 to 90.2% in November 20233 (human passing score is 60%, human expert score is 87%4).”
From AgentClinic: a multimodal benchmark for tool-using clinical AI agents (2026) · cites this paper as [2]
Experimental — accuracy not yet measured
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“State-of-the-art neural languagemodels are generally based on transformer architectures [53], following the recent success of BERT [16, 38].”
BackgroundHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“models in GPT [47, 48] and BERT [16, 38]. Their success is fueled by access to large text corpora, advanced hardware such as GPUs, and a culmination of advances in optimization methods, such as Adam [29] and slanted triangular learning rate [22]. Here, transfer learning goes from the pretrained language models to fine…”
MethodInfluentialExploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 5×
“nto question [38], but we include it in our pretraining experiments to enable a head-to-head comparison with prior BERT models. 2.4 Advanced Pretraining Techniques In the original formulation of BERT [16], the masked language model (MLM) simply selects random subwords to mask. When a word is only partially masked, it…”
BackgroundMethodInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“This starts with simple stand-alone word embeddings [40, 45], and evolves into more sophisticated pretrained language models, from LSTM in ULMFiT [22] and ELMo [46] to transformer-based models in GPT [47, 48] and BERT [16, 38].”
MethodHanahan, D., & Weinberg, R. A. (2000) · Cell · doi:10.1016/s0092-8674(00)81683-9 · Cited by 29,219
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“This starts with simple stand-alone word embeddings [40, 45], and evolves into more sophisticated pretrained language models, from LSTM in ULMFiT [22] and ELMo [46] to transformer-based models in GPT [47, 48] and BERT [16, 38].”
MethodRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 7×
“For example, the original BERT model was trained on Wikipedia1 and BookCorpus [59], and subsequent efforts have focused on crawling additional text from the Web to power even larger-scale pretraining [38, 49].”
BackgroundMethodInfluentialConditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Research works printed but not linked: 14
- Marianna Apidianaki, Saif M. Mohammad, Jonathan May, Ekaterina Shutova, Steven Bethard, and Marine Carpuat (Eds.). 2018. Proceedings of the 12th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018. Association for Computational Linguistics. https://www.aclweb.org/anthology/volumes/S18-1/.
- 10.5555/2145432.2145474 · doi:10.5555/2145432.2145474
- Steven Bethard, Marine Carpuat, Marianna Apidianaki, Saif M. Mohammad, Daniel M. Cer, and David Jurgens (Eds.). 2017. Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval@ACL 2017, Vancouver, Canada, August 3-4, 2017. Association for Computational Linguistics. https://www.aclweb.org/anthology/volumes/S17-2/.
- Steven Bethard, Daniel M. Cer, Marine Carpuat, David Jurgens, Preslav Nakov, and Torsten Zesch (Eds.). 2016. Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2016, San Diego, CA, USA, June 16-17, 2016. Association for Computational Linguistics. https://www.aclweb.org/anthology/volumes/S16-1/.
- Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, and Junichi Tsujii (Eds.). 2019. Proceedings of the 18th BioNLP Workshop and Shared Task, BioNLP@ACL 2019, Florence, Italy, August 1, 2019. Association for Computational Linguistics. https://www.aclweb.org/anthology/volumes/W19-50/.
- Mona T. Diab, Timothy Baldwin, and Marco Baroni (Eds.). 2013. Proceedings of the 7th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2013, Atlanta, Georgia, USA, June 14-15, 2013. Association for Computational Linguistics. https://www.aclweb.org/anthology/volumes/S13-2/.
- 10.5555/2598938.2599127 · doi:10.5555/2598938.2599127
- 10.5555/1567594.1567610 · doi:10.5555/1567594.1567610
- 10.5555/2107691.2107693 · doi:10.5555/2107691.2107693
- Sun Kim, Rezarta Islamaj Dogan, Andrew Chatr-aryamontri, Mike Tyers, W. John Wilbur, and Donald C. Comeau. 2015. Overview of BioCreative V BioC track. In Proceedings of the Fifth BioCreative Challenge Evaluation Workshop, Sevilla, Spain. 1–9.
- Martin Krallinger, Obdulia Rabal, Saber A. Akhondi, Martın Pérez Pérez, Jesús Santamaría, G. P. Rodríguez, G. Tsatsaronis, et al. 2017. Overview of the BioCreative VI Chemical-Protein Interaction Track. In Proceedings of the 6th BioCreative Challenge Evaluation Workshop, Vol. 1. 141–146.
- 10.5555/645530.655813 · doi:10.5555/645530.655813
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.
- 10.5555/3454287.3454581 · doi:10.5555/3454287.3454581
Cited by
Items 101–200 of 1,030
A comprehensive evaluation of large Language models on benchmark biomedical text processing tasks
Jahan, I., Laskar, M. T. R., Peng, C., & Huang, J. X. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108189 · Cited by 189
BioELECTRA:Pretrained Biomedical text Encoder using Discriminators
Kanakarajan, K. R., Kundumani, B., & Sankarasubbu, M. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.16 · Cited by 181
A multimodal visual–language foundation model for computational ophthalmology
Shi, D., Zhang, W., Yang, J., Huang, S., Chen, X., Xu, P., Jin, K., Lin, S. C., Wei, J., Yusufu, M., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01772-2 · Cited by 161
Neural Approaches to Conversational Information Retrieval
Gao, J., Xiong, C., Bennett, P. N., & Craswell, N. (2023) · The information retrieval series · doi:10.1007/978-3-031-23080-6 · Cited by 146
Self-citationAn extensive benchmark study on biomedical text generation and mining with ChatGPT
Chen, Q., Sun, H., Liu, H., Jiang, Y., Ran, T., Jin, X., Xiao, X., Lin, Z., Chen, H., & Niu, Z. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad557 · Cited by 97
“PubmedBERT (Gu et al. 2021) was pretrained on PubMed corpus while BioLinkBERT-Base (Yasunaga et al. 2022) Table 3.”
MethodGenerative Artificial Intelligence Through ChatGPT and Other Large Language Models in Ophthalmology
Tan, T. F., Thirunavukarasu, A. J., Campbell, J. P., Keane, P. A., Pasquale, L. R., Abràmoff, M. D., Kalpathy–Cramer, J., Lum, F., Kim, J. E., Baxter, S. L., et al. (2023) · Ophthalmology Science · doi:10.1016/j.xops.2023.100394 · Cited by 144
TLT: Recurrent fine-tuning transfer learning for water quality long-term prediction
Lin, P., Wu, H., Gao, M., Yi, H., Xiong, Q., Yang, Y., & Cheng, S. (2022) · Water Research · doi:10.1016/j.watres.2022.119171 · Cited by 124
Transformer-based models for chemical SMILES representation: A comprehensive literature review
Mswahili, M. E., & Jeong, Y.-S. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e39038 · Cited by 142
Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images
Lu, M. Y., Chen, B., Zhang, A., Williamson, D. F. K., Chen, R. J., Ding, T., Le, L. P., Chuang, Y.-S., & Mahmood, F. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01893 · Cited by 122
“[23] Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon.”
BackgroundMethodInfluentialCODER: Knowledge-infused cross-lingual medical term embedding for term normalization
Yuan, Z., Zhao, Z., Sun, H., Li, J., Wang, F., & Yu, S. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103983 · Cited by 111
Huang, Y., Gomaa, A., Semrau, S., Haderlein, M., Lettmaier, S., Weißmann, T., Grigo, J., Tkhayat, H. B., Frey, B., Gaipl, U. S., et al. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1265024 · Cited by 131
“Several domain-specific language models have been developed such as BioBERT (10), PubMedBERT (9), and ClinicalBERT (12).”
BackgroundRadiology-GPT: A large language model for radiology
Liu, Z., Li, Y., Shu, P., Zhong, A., Jiang, H., Pan, Y., Yang, L., Ju, C., Wu, Z., Ma, C., et al. (2025) · Meta-Radiology · doi:10.1016/j.metrad.2025.100153 · Cited by 133
Zhou, S., Wang, N., Wang, L., Liu, H., & Zhang, R. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac040 · Cited by 119
PLM-ICD: Automatic ICD Coding with Pretrained Language Models
Huang, C.-W., Tsai, S.-C., & Chen, Y.-N. (2022) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2022.clinicalnlp-1.2 · Cited by 189
Comprehensive echocardiogram evaluation with view primed vision language AI
Vukadinovic, M., Chiu, I.-M., Tang, X., Yuan, N., Chen, T., Cheng, P. P., Li, D., Cheng, S. C., He, B. D., & Ouyang, D. (2025) · Nature · doi:10.1038/s41586-025-09850-x · Cited by 145
“For the text encoder, we selected Wordpiece Encoding tokenization and BERT 37 architecture with weights pretrained on PubMed abstracts (BioMed-BERT 38 ).”
Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review
Neha, F., Bhati, D., & Shukla, D. K. (2025) · AI · doi:10.3390/ai6090226 · Cited by 117
“These systems use biomedical embeddings (e.g., BioBERT, PubMedBERT [30]), specialized corpora (e.g., PubMed abstracts and clinical notes), and domain-specific knowledge bases (e.g., unified medical language system (UMLS), MIMIC).”
Artificial intelligence: revolutionizing cardiology with large language models
Boonstra, M., Weissenbacher, D., Moore, J. H., Gonzalez‐Hernandez, G., & Asselbergs, F. W. (2023) · European Heart Journal · doi:10.1093/eurheartj/ehad838 · Cited by 132
“Both generic LLMs, like BERT, 42 PaLM, 7 BLOOM, 43 or LLaMa, 44 and domain (Med-PaLM 45 and PubMedBERT 46 ) and language-specific (MedRoBERTa.nl 47 ) models are used for various tasks, for example to provide an overview of relevant medical literature 48 (evidencehunt.com).”
XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameters
Zhang, X., & Yang, Q. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615285 · Cited by 199
“For example, BioBERT [5] and PubMedBERT [4] are proposed for the biomedical field, and BloombergGPT [19] are proposed for financial scenarios.”
BackgroundGeneAgent: self-verification language agent for gene-set analysis using domain databases
Wang, Z., Jin, Q., Wei, C.-H., Tian, S., Lai, P., Zhu, Q., Day, C., Ross, C., Leaman, R., & Lu, Z. (2025) · Nature Methods · doi:10.1038/s41592-025-02748-6 · Cited by 158
Gangwal, A., & Lavecchia, A. (2025) · Drug Discovery Today · doi:10.1016/j.drudis.2025.104360 · Cited by 129
Large language models for chemistry robotics
Yoshikawa, N., Skreta, M., Darvish, K., Arellano-Rubach, S., Ji, Z., Bjørn Kristensen, L., Li, A. Z., Zhao, Y., Xu, H., Kuramshin, A., et al. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10136-2 · Cited by 112
“Second, LLMs may perform poorly in generating task plans in a zero-shot manner for data-scarce domain-specific languages, such as chemistry (Gu et al., 2021; Liu et al., 2022).”
MethodThe Diagnostic and Triage Accuracy of the GPT-3 Artificial Intelligence Model
Levine, D. M., Tuwani, R., Kompa, B., Varma, A., Finlayson, S. G., Mehrotra, A., & Beam, A. (2023) · doi:10.1101/2023.01.30.23285067 · Cited by 106
“There is a large body of literature demonstrating language models such as GPT-3 result in significant improvements for pre-processing biomedical text 2,31–35 and for prediction of important outcomes like disease onset 12,36 , mortality 37 , and readmission 38 .”
BackgroundLiu, F., Vulić, I., Korhonen, A., & Collier, N. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.109 · Cited by 162
Reichenpfader, D., Müller, H., & Denecke, K. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01219-0 · Cited by 138
Mining legal arguments in court decisions
Habernal, I., Faber, D. R., Recchia, N., Bretthauer, S., Gurevych, I., Döhmann, I. S. G., & Burchard, C. (2023) · Artificial Intelligence and Law · doi:10.1007/s10506-023-09361-y · Cited by 108
Pei, Q., Zhang, W., Zhu, J., Wu, K., Gao, K., Wu, L., Xia, Y., & Yan, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.70 · Cited by 187
Recent Advances in Large Language Models for Healthcare
Nassiri, K., & Akhloufi, M. A. (2024) · BioMedInformatics · doi:10.3390/biomedinformatics4020062 · Cited by 133
“Previously, most works relied on smaller linguistic models trained specifically on domain-specific healthcare text corpora [94,157–159].”
BackgroundAutomated Radiology Report Generation: A Review of Recent Advances
Sloan, P., Clatworthy, P. L., Simpson, E., & Mirmehdi, M. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3408456 · Cited by 169
Improving Biomedical Pretrained Language Models with Knowledge
Yuan, Z., Liu, Y., Tan, C., Huang, S., & Huang, F. (2021) · Proceedings of the 20th Workshop on Biomedical Language Processing · doi:10.18653/v1/2021.bionlp-1.20 · Cited by 79
Zhang, G., Jin, Q., Zhou, Y., Wang, S., Idnay, B., Luo, Y., Park, E., Nestor, J. G., Spotnitz, M. E., Soroush, A., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01239-w · Cited by 131
Advancing entity recognition in biomedicine via instruction tuning of large language models
Keloth, V. K., Hu, Y., Xie, Q., Peng, X., Wang, Y., Zheng, A., Selek, M., Raja, K., Wei, C. H., Jin, Q., et al. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae163 · Cited by 104
“This is consistent with the original PubMedBERT study (Gu et al. 2021).”
MethodResultInfluentialData extraction from polymer literature using large language models
Gupta, S., Mahmood, A. U., Shetty, P., Adeboye, A., & Ramprasad, R. (2024) · Communications Materials · doi:10.1038/s43246-024-00708-9 · Cited by 126
PathAsst: A Generative Foundation AI Assistant towards Artificial General Intelligence of Pathology
Sun, Y., Zhu, C., Zheng, S., Zhang, K., Sun, L., Shui, Z., Zhang, Y., Li, H., & Yang, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28308 · Cited by 158
“We utilize PubMedBERT (Gu et al. 2021) for abstract embedding extraction and Faiss (Johnson, Douze, and J´egou 2019) for the efficient storage of these embeddings.”
MethodDRG-LLaMA : tuning LLaMA model to predict diagnosis-related group for hospitalized patients
Wang, H., Gao, C., Dantona, C., Hull, B., & Sun, J. (2024) · npj Digital Medicine · doi:10.1038/s41746-023-00989-3 · Cited by 99
“Large language model context: Language models based on the transformer architecture, either pretrained or fine-tuned using biomedical corpora, have demonstrated efficacy across a spectrum of NLP benchmarks within the biomedical realm (Lee et al. 2020; Huang, Altosaar, and Ranganath 2020; Gu Toward deploying a local LL…”
Kumar, K., Kumar, P., Deb, D., Ungureșan, M.-L., & Mureşan, V. (2023) · Healthcare · doi:10.3390/healthcare11020207 · Cited by 122
Pre-trained language models with domain knowledge for biomedical extractive summarization
Xie, Q., Bishop, J. A., Tiwari, P., & Ananiadou, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109460 · Cited by 120
“Meng et al. [29] proposed to partition the knowledge graph into subgraphs and infused them into a series of PLMs including, BioBERT, SciBERT, and PubmedBERT [30].”
MethodInfluentialClinical entity augmented retrieval for clinical information extraction
López, I., Swaminathan, A., Vedula, K. S., Narayanan, S., Haredasht, F. N., Ma, S. P., Liang, A. S., Tate, S., Maddali, M. V., Gallo, R. J., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01377-1 · Cited by 148
Redefining biomaterial biocompatibility: challenges for artificial intelligence and text mining
Mateu‐Sanz, M., Fuenteslópez, C. V., Uribe-Gomez, J., Haugen, H. J., Pandit, A. S., Ginebra, M., Hakimi, O., Krallinger, M., & SAMARA, A. A. (2023) · Trends in biotechnology · doi:10.1016/j.tibtech.2023.09.015 · Cited by 95
Application of large language models in medicine
Liu, F., Zhou, H., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S. S., Hua, Y., Zhou, P., et al. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00279-5 · Cited by 94
The importance of resource awareness in artificial intelligence for healthcare
Jia, Z., Chen, J., Xu, X., Kheir, J., Hu, J., Xiao, H., Peng, S., Hu, X. S., Chen, D., & Shi, Y. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00670-0 · Cited by 86
A Large Language Model Approach to Educational Survey Feedback Analysis
Parker, M. J., Anderson, C., Stone, C., & Oh, Y. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-024-00414-0 · Cited by 124
“Other studies ([28, 29]) demonstrate the potential for ChatGPT to perform text annotation or provide natural language explanations at levels approaching or matching those of humans.”
A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification
Wang, D., Wang, X., Wang, L., Li, M., Da, Q., Liu, X., Gao, X., Shen, J., He, J., Tian, S., et al. (2023) · Scientific Data · doi:10.1038/s41597-023-02460-0 · Cited by 129
Self-citationArtificial General Intelligence for Medical Imaging Analysis
Li, X., Zhao, L., Zhang, L., Wu, Z., Liu, Z., Jiang, H., Cao, C., Xu, S., Li, Y., Dai, H., et al. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3493775 · Cited by 116
“• Tailoring to the Medical Domain: It is important to tailor language models to specific domains [62]–[64].”
CHEAT: A Large-Scale Dataset for Detecting CHatGPT-writtEn AbsTracts
Yu, P., Chen, J., Feng, X., & Xia, Z. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3536929 · Cited by 42
The landscape of biomedical research
González-Márquez, R., Schmidt, L., Schmidt, B. M., Berens, P., & Kobak, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100968 · Cited by 89
“…chemistry, computation, dermatology, ecology, education, engineering, environment, ethics, genetics, gynecology, healthcare, immunology, infectious, material, microbiology, neurology, neuroscience, nursing, nutrition, ophthalmology, optics, pathology, pediatric, pharmacology, physics, physiology,…”
InfluentialTai, W. K., Kung, H. T., Dong, X., Comiter, M. Z., & Kuo, C. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.129 · Cited by 125
A survey on retrieval-augmentation generation (RAG) models for healthcare applications
El-Enen, M. A. A., Saad, S., & Nazmy, T. M. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11666-9 · Cited by 118
Kolides, A., Nawaz, A., Rathor, A., Beeman, D., Hashmi, M., Fatima, S., Berdik, D., Al‐Ayyoub, M., & Jararweh, Y. (2023) · Simulation Modelling Practice and Theory · doi:10.1016/j.simpat.2023.102754 · Cited by 101
Bereczki, Z., Benczik, B., Balogh, O. M., Marton, S., Puhl, E., Pétervári, M., Váczy‐Földi, M., Papp, Z. T., Makkos, A., Glass, K., et al. (2024) · British Journal of Pharmacology · doi:10.1111/bph.17302 · Cited by 94
“…(BERT) model (Devlin et al., 2019), its improved successor RoBERTa (Y. Liu et al., 2019) and their numerous domain-specific versions (BioBERT [J. Lee et al., 2020], PubMedBERT [T. Gu et al., 2021] and BioMed-RoBERTa [Gururangan et al., 2020]) that were pre-trained on biomedical literature.”
Systematic Analysis of Retrieval-Augmented Generation-Based LLMs for Medical Chatbot Applications
Bora, A., & Cuayáhuitl, H. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6040116 · Cited by 105
Has multimodal learning delivered universal intelligence in healthcare? A comprehensive survey
Lin, Q., Zhu, Y., Mei, X., Huang, L., Ma, J., He, K., Peng, Z., Cambria, E., & Feng, M. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102795 · Cited by 115
Information Retrieval: Advanced Topics and Techniques
(2024) · ACM eBooks · doi:10.1145/3674127 · Cited by 88
Does the magic of BERT apply to medical code assignment? A quantitative study
Ji, S., Hölttä, M., & Marttinen, P. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104998 · Cited by 108
Task-Specific Transformer-Based Language Models in Health Care: Scoping Review
Cho, H. N., Jun, T. J., Kim, Y., Kang, H. J., Ahn, I., Gwon, H., Kim, Y., Seo, H., Choi, H., Kim, M., et al. (2024) · JMIR Medical Informatics · doi:10.2196/49724 · Cited by 106
Open challenges and opportunities in federated foundation models towards biomedical healthcare
Li, X., Peng, L., Wang, Y., & Zhang, W. (2025) · BioData Mining · doi:10.1186/s13040-024-00414-9 · Cited by 105
“Foundation models can greatly benefit from training on expanded, domain-specific corpora [183].”
InfluentialFraile Navarro, D., Ijaz, K., Rezazadegan, D., Rahimi-Ardabili, H., Dras, M., Coiera, E., & Berkovsky, S. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105122 · Cited by 82
Yang, H., Li, M., Zhou, H., Xiao, Y., Fang, Q., Zhou, S., & Zhang, R. (2025) · Journal of Medical Internet Research · doi:10.2196/70080 · Cited by 60
“This integration must consider factors such as the compatibility of different models, the method of aggregating their outputs, and maintaining the interpretability of the responses [42-46].”
AIONER: all-in-one scheme-based biomedical named entity recognition using deep learning
Luo, L., Wei, C.-H., Lai, P., Leaman, R., Chen, Q., & Lu, Z. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad310 · Cited by 99
Context Matters: A Strategy to Pre-train Language Model for Science Education
Liu, Z., He, X., Liu, L., Liu, T., & Zhaı, X. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-36336-8_103 · Cited by 93
“Studies have shown that the performance on NLP tasks can be improved by using domain-specific data to contextualize language models [5].”
BackgroundInfluentialCurrent and future state of evaluation of large language models for medical summarization tasks
Croxford, E., Gao, Y., Pellegrino, N., Wong, K. K. Y., Wills, G. J., First, E., Liao, F., Goswami, C., Patterson, B. W., & Afshar, M. (2025) · npj Health Systems · doi:10.1038/s44401-024-00011-2 · Cited by 108
BioM-Transformers: Building Large Biomedical Language Models with BERT, ALBERT and ELECTRA
Alrowili, S., & Shanker, V. K. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.24 · Cited by 117
Extending the Frontier of ChatGPT: Code Generation and Debugging
Sakib, F. A., Khan, S. H., & Karim, A. (2024) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet61485.2024.10698405 · Cited by 100
“and customer service [1], [2], [7]–[9].”
Terranova, N., Renard, D., Shahin, M. H., Menon, S., Cao, Y., Hop, C. E. C. A., Hayes, S., Madrasi, K., Stodtmann, S., Tensfeldt, T., et al. (2024) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3053 · Cited by 78
“Several biomedical and clinical domain-specific LLMs have been developed in the past few years, such as PubMedBERT, 71 BioBERT, 72 Med-BERT BERT, 73 and ClinicalBERT.”
MethodUnsupervised and self-supervised deep learning approaches for biomedical text mining
Nadif, M., & Role, F. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab016 · Cited by 85
Incorporating medical knowledge in BERT for clinical relation extraction
Roy, A., & Pan, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.435 · Cited by 102
Fast changeset-based bug localization with BERT
Ciborowska, A., & Damevski, K. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510042 · Cited by 60
“Researchers in the biomedical domain recognized the same issue affecting medical terms and proposed domain-specific BERT adaptations [2, 11, 26].”
BackgroundmedBERT.de: A comprehensive German BERT model for the medical domain
Bressem, K. K., Papaioannou, J.-M., Grundmann, P., Borchert, F., Adams, L. C., Liu, L., Busch, F., Xu, L., Loyen, J. P., Niehues, S. M., et al. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121598 · Cited by 87
“However, even within a language, technical languages can be very different from spoken language, necessitating the development of domain-specific models [10].”
BackgroundMethodFrisoni, G., Moro, G., & Carbonaro, A. (2021) · IEEE Access · doi:10.1109/access.2021.3130956 · Cited by 84
“, SciBERT [268], BioBERT [269], ELECTRAMed [270], PubMedBERT [271], and SciFive [272] for biomedicine), some researchers have investigated their use as backbones for EE.”
BackgroundHe, Z., Bhasuran, B., Jin, Q., Tian, S., Hanna, K., Shavor, C., Arguello, L. G., Murray, P. R., & Lu, Z. (2024) · Journal of Medical Internet Research · doi:10.2196/56655 · Cited by 92
“…Encoder Representations from Transformers [SciBERT] [34], and PubMed-trained Bidirectional Encoder Representations from Transformers [PubMedBERT] [35]) and various automated machine learning (autoML) models (XGBoost, NeuralNet, CatBoost, weighted ensemble, and LightGBM) to automatically identify…”
From screens to scenes: A survey of embodied AI in healthcare
Liu, Y., Cao, X., Chen, T., Jiang, Y., You, J., Wu, M., Wang, X., Feng, M., Jin, Y., & Chen, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103033 · Cited by 92
An Overview of Language Models: Recent Developments and Outlook
Wei, C., Wang, Y.-C., Wang, B., & Kuo, C. ‐. J. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000010 · Cited by 84
“In contrast, domain-specific LMs [160, 161, 162, 163] are designed to deal with domain-specific tasks, e.”
BackgroundHuman–AI collectives most accurately diagnose clinical vignettes
Zöller, N., Berger, J., Lin, I., Fu, N., Komarneni, J., Barabucci, G., Laskowski, K., Shia, V., Harack, B., Chu, E. A., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2426153122 · Cited by 98
Me-LLaMA: Foundation Large Language Models for Medical ApplicationsPreprint
Xie, Q., Chen, Q., Chen, A., Cheng, P., Hu, Y., Lin, F., Peng, X., Huang, J., Zhang, J., Keloth, V. K., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4240043/v1 · Cited by 103
“Due to privacy concerns, which preclude the transmission of clinical datasets with patient information to ChatGPT and GPT-4, we conducted our comparison across eight datasets that are not subject to these limitations, including PubMedQA, MedQA, MedMCQA, HoC, MTsample, PubMed, BioNLI and 2013 DDI.”
InfluentialEvaluating GPT and BERT models for protein–protein interaction identification in biomedical text
Rehana, H., Çam, N. B., Basmaci, M., Zheng, J., Jemiyo, C., He, Y. O., Özgür, A., & Hur, J. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae133 · Cited by 78
“5 and GPT-4 variants, against masked language models like BioBERT, PubMedBERT, and SciBERT in a PPI prediction task.”
BackgroundMethodInfluentialPharmaBench: Enhancing ADMET benchmarks with large language models
Niu, Z., Xiao, X., Wu, W., Cai, Q., Jiang, Y., Jin, W., Wang, M., Yang, G., Kong, L., Jin, X., et al. (2024) · Scientific Data · doi:10.1038/s41597-024-03793-0 · Cited by 93
“Recently developed Large Language Models (LLMs) like ChatGPT 17 , PubMedBERT 18 , and BioBERT 19 represent a novel approach of effectively extracting data from a large body of text, therefore a potential method for addressing data curation challenges.”
MethodBenchmarking for biomedical natural language processing tasks with a domain specific ALBERT
Naseem, U., Dunn, A. G., Khushi, M., & Kim, J. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04688-w · Cited by 67
“[14] developed PubMedBERT by training from scratch on PubMed articles and showed performance gained over models trained on general corpora.”
MethodInfluentialReale-Nosei, G., Amador-Domínguez, E., & Serrano, E. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103264 · Cited by 88
Progress and opportunities of foundation models in bioinformatics
Li, Q., Hu, Z., Wang, Y., Li, L., Fan, Y., King, I., Jia, G., Wang, S., Song, L., & Li, Y. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae548 · Cited by 88
“Biological FMs like GMAI [14], BLURB [25], and Enformer [34] can be fine-tuned with just a few examples, making this approach both efficient and effective for tasks where obtaining large datasets is challenging.”
BackgroundMethodInfluentialTowards BERT-based Automatic ICD Coding: Limitations and Opportunities
Pascual, D., Luck, S., & Wattenhofer, R. P. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.6 · Cited by 86
Zhang, Y., Sui, X., Pan, F., Yu, K., Li, K., Tian, S., Erdengasileng, A., Han, Q., Wang, W., Wang, J., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01014-w · Cited by 101
Natural Language Processing in medicine and ophthalmology: A review for the 21st-century clinician
Rojas-Carabali, W. A., Agrawal, R., Gutiérrez-Sinisterra, L., Baxter, S. L., Cifuentes‐González, C., Wei, Y. C., Abisheganaden, J. A., Kannapiran, P., Wong, S., Lee, B. T. K., et al. (2024) · Asia-Pacific Journal of Ophthalmology · doi:10.1016/j.apjo.2024.100084 · Cited by 82
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2024) · doi:10.2139/ssrn.4809363 · Cited by 69
Large language models for biomedicine: foundations, opportunities, challenges, and best practices
Sahoo, S. S., Plasek, J. M., Xu, H., Uzuner, Ö., Cohen, T., Yetişgen, M., Liu, H., Meystre, S. M., & Wang, Y. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae074 · Cited by 80
Large Language Models in Medical Diagnostics: Scoping Review With Bibliometric Analysis
Su, H., Sun, Y., Li, R., Zhang, A., Yang, Y., Xiao, F., Duan, Z., Chen, J., Hu, Q., Yang, T., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/72062 · Cited by 79
Biomedical Named Entity Recognition at Scale
Kocaman, V., & Talby, D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68763-2_48 · Cited by 96
Large language models in psychiatry: Opportunities and challenges
Volkmer, S., Meyer-Lindenberg, A., & Schwarz, E. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.116026 · Cited by 59
Development of a large-scale medical visual question-answering dataset
Zhang, X., Wu, C., Zhao, Z., Lin, W., Zhang, Y., Wang, Y., & Xie, W. (2024) · Communications Medicine · doi:10.1038/s43856-024-00709-2 · Cited by 114
Structured information extraction from complex scientific text with fine-tuned large language models
Dunn, A., Dagdelen, J., Walker, N., Lee, S., Rosen, A. S., Ceder, G., Persson, K., & Jain, A. (2022) · arXiv · doi:10.48550/arxiv.2212.05238 · Cited by 67
“seq2rel to perform sentence-level extraction with 30 epochs, batch size of 4, encoder learning rate 2 · 10−5, decoder learning rate 5 · 10−4, and pretrained BiomedNLP BERT tokenizer[44] (further training details are in Supporting Information).”
MethodA sequence-to-sequence approach for document-level relation extraction
Giorgi, J., Bader, G., & Wang, B. (2022) · Proceedings of the 21st Workshop on Biomedical Language Processing · doi:10.18653/v1/2022.bionlp-1.2 · Cited by 58
A Survey on Relation Extraction
Detroja, K., Bhensdadia, C. K., & Bhatt, B. (2022) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200244 · Cited by 113
A trimodal protein language model enables advanced protein searches
Su, J., He, Y., You, S., Jiang, S. J., Zhou, X., Zhang, X., Wang, Y., Su, X., Tolstoy, I., Chang, X., et al. (2025) · Nature Biotechnology · doi:10.1038/s41587-025-02836-0 · Cited by 81
“The architecture of ProTrek incorporates a pretrained evolutionary-scale modeling (ESM) encoder 18 for amino acid sequence modeling and a pretrained BERT (bidirectional encoder representations from transformers) 25 for nuanced natural language function representation.”
RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
Jain, S., Agrawal, A., Saporta, A., Truong, S. Q., Duong, D. N., Bui, T., Chambon, P., Zhang, Y., Lungren, M. P., Ng, A. Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.14463 · Cited by 66
“Biomedical pretraining For each of our approaches, in addition to using BERT weight initializations, we use weight initializations from four different biomedical pretrained models, which are BioBERT [33], Bio+ClinicalBERT [34], PubMedBERT [35], and BlueBERT [36].”
MethodCorrectness Comparison of ChatGPT ‐4, Gemini, Claude‐3, and Copilot for Spatial Tasks
Hochmair, H. H., Juhász, L., & Kemp, T. (2024) · Transactions in GIS · doi:10.1111/tgis.13233 · Cited by 82
Enhancing knowledge retrieval with in-context learning and semantic search through generative AI
Ghali, M.-K., Farrag, A., Won, D., & Jin, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113047 · Cited by 77
“In contrast to prevailing assumptions, [1] highlights significant benefits of starting domain-specific language models from scratch, particularly in biomedicine.”
ThoughtSource: A central hub for large language model reasoning data
Ott, S., Hebenstreit, K., Liévin, V., Hother, C. E., Moradi, M., Mayrhauser, M., Praas, R., Winther, O., & Samwald, M. (2023) · Scientific Data · doi:10.1038/s41597-023-02433-3 · Cited by 80
Pretrained Biomedical Language Models for Clinical NLP in Spanish
Carrino, C. P., Palao, J. L., Pàmies, M., Gutiérrez-Fandiño, A., Armengol-Estapé, J., Silveira-Ocampo, J., Valencia, A., González-Agirre, A., & Villegas, M. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.19 · Cited by 99
“Specifically, in the biomedical domain, there has been a proliferation of models (Peng et al., 2019; Beltagy et al., 2019; Alsentzer et al., 2019; Gu et al., 2021) since the first BioBERT (Lee et al., 2019) model was published.”
BackgroundMethodAgriBERT: Knowledge-Infused Agricultural Language Models for Matching Food and Nutrition
Rezayi, S., Liu, Z., Wu, Z., Dhakal, C., Ge, B., Zhen, C., Liu, T., & Li, S. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/715 · Cited by 97
Data-Centric Foundation Models in Computational Healthcare: A Survey
Zhang, Y., Gao, J. G., Tan, Z., Zhou, L., Ding, K., Zhou, M., Zhang, S., & Wang, D. (2026) · ACM Computing Surveys · doi:10.1145/3800677 · Cited by 74
“Researchers have established benchmarks for model evaluation in the healthcare field, inspired by the progress of AI research in vision and language domains [79, 105, 243].”
InfluentialSelf-citationHolste, G., Zhou, Y., Wang, S., Jaiswal, A. K., Lin, M., Zhuge, S., Yang, Y., Kim, D., Nguyen-Mau, T.-H., Tran, M., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103224 · Cited by 83
“The authors utilized an ImageNet-pretrained ResNet50 (He et al., 2016) and text encoder pre-trained on PubMedBERT Table 1: Overview of top-performing CXR-LT challenge solutions.”
InfluentialRezayi, S., Liu, Z., Wu, Z., Dhakal, C., Ge, B., Dai, H., Mai, G., Liu, N., Zhen, C., Liu, T., et al. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2024.3442542 · Cited by 79
“In other words, pre-training BERT models with in-domain corpora can significantly improve overall performance on a wide variety of downstream tasks [24].”
Influential
Metadata
- Authors
- Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H.
- Affiliations
- Microsoft Research, Redmond, WA
- Microsoft (United States) (US)
- ORCID
- Venue
- ACM Transactions on Computing for Healthcare
- Published
- 2021-10-15
- Volume
- 3
- Issue
- 1
- Pages
- 1-23
- ISSN
- 2637-8051 (ISSN-L)
- 2691-1957
- OpenAlex
- W3046375318
- Cited by
- 2,170
- References
- 63
- Citations by year
- 2026: 402 · 2025: 512 · 2024: 456 · 2023: 411 · 2022: 243 · 2021: 125
- FWCI
- 109.37
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Transactions on Computing for Healthcare (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (CNS-1738645 and DRL-1837446)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- biomedical natural language processing, BERT, fine-tuning, named entity recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.