Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets
Open accessAbstract
Inspired by the success of the General Language Understanding Evaluation benchmark, we introduce the Biomedical Language Understanding Evaluation (BLUE) benchmark to facilitate research in the development of pre-training language representations in the biomedicine domain.The benchmark consists of five tasks with ten datasets that cover both biomedical and clinical texts with different dataset sizes and difficulties.We also evaluate several baselines based on BERT and ELMo and find that the BERT model pre-trained on PubMed abstracts and MIMIC-III clinical notes achieves the best results.We make the datasets, pre-trained models, and codes publicly available at https://github.com/ ncbi-nlp/BLUE_Benchmark.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
BERT demonstrated promising results in several clinical NLP tasks.
“and demonstrated promising results in several clinical NLP tasks [27-29].”
From Extracting Family History of Patients From Clinical Narratives: Exploring an End-to-End Solution With Deep Learning Models (2020) · cites this paper as [29]
Kumar et al. [102] and Peng et al. [103] achieved an F1 of 0.8501.
“| Kumar et al. [102], Peng et al. [103] | Word embedding | Text | witter self-report | F1: 0.8501 |”
From A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis (2024) · cites this paper as [103]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · see citing works
Neural Architectures for Named Entity Recognition
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1030 · Cited by 4,401 · see citing works
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · see citing works
Jurafsky, D. S., & Martin, J. (2000) · Cited by 4,132 · see citing works
A Review on Multi-Label Learning Algorithms
Zhang, M.-L., & Zhou, Z. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2013.39 · Cited by 3,336 · see citing works
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works
Alsentzer, E., Murphy, J., Boag, W., Weng, W.-H., Jindi, D., Naumann, T., & McDermott, M. (2019) · Proceedings of the 2nd Clinical Natural Language Processing Workshop · doi:10.18653/v1/w19-1909 · Cited by 1,736 · see citing works
(2018) · International Conference on Learning Representations · Cited by 1,621 · see citing works
2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text
Uzuner, Ö., South, B. R., Shen, S., & DuVall, S. L. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000203 · Cited by 1,258 · see citing works
BioCreative V CDR task corpus: a resource for chemical disease relation extraction
Li, J., Sun, Y.-P., Johnson, R. J., Sciaky, D., Wei, C.-H., Leaman, R., Davis, A., Mattingly, C., Wiegers, T. C., & Lu, Z.-Y. (2016) · Database J. Biol. Databases Curation · doi:10.1093/database/baw068 · Cited by 1,115 · see citing works
Self-citationPublicly Available Clinical BERT EmbeddingsPreprint
Alsentzer, E., Murphy, J. R., Boag, W., Weng, W., Jin, D., Naumann, T., & McDermott, M. B. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03323 · Cited by 1,040 · see citing works
Semi-supervised sequence tagging with bidirectional language models
Peters, M., Ammar, W., Bhagavatula, C., & Power, R. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1161 · Cited by 655 · see citing works
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2018) · arXiv · doi:10.48550/arxiv.1804.07461 · Cited by 562 · see citing works
The DDI corpus: An annotated corpus with pharmacological substances and drug–drug interactions
Herrero-Zazo, M., Segura-Bédmar, I., Martı́nez, P., & Declerck, T. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.07.011 · Cited by 479 · see citing works
BioWordVec, improving biomedical word embeddings with subword information and MeSH
Zhang, Y., Chen, Q., Yang, Z., Lin, H., & Lu, Z. (2019) · Scientific Data · doi:10.1038/s41597-019-0055-0 · Cited by 455 · see citing works
Self-citationA comparison of word embeddings for the biomedical natural language processing
Wang, Y., Liu, S., Afzal, N., Rastegar-Mojarad, M., Wang, L., Shen, F., Kingsbury, P., & Liu, H. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.09.008 · Cited by 370 · see citing works
How to Train good Word Embeddings for Biomedical NLP
Chiu, B., Crichton, G., Korhonen, A., & Pyysalo, S. (2016) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/w16-2922 · Cited by 344 · see citing works
Enhancing clinical concept extraction with contextual embeddings
Si, Y., Wang, J., Xu, H., & Roberts, K. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz096 · Cited by 322 · see citing works
Overview of the ShARe/CLEF eHealth Evaluation Lab 2013
Suominen, H., Salanterä, S., Velupillai, S. U., Chapman, W. W., Savova, G., Elhadad, N., Pradhan, S., South, B. R., Mowery, D. L., Jones, G. J. F., et al. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40802-1_24 · Cited by 255 · see citing works
Lessons from Natural Language Inference in the Clinical Domain
Romanov, A., & Shivade, C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1187 · Cited by 226 · see citing works
Community challenges in biomedical text mining over 10 years: success, failure and the future
Huang, C., & Lu, Z. (2015) · Briefings in Bioinformatics · doi:10.1093/bib/bbv024 · Cited by 225 · see citing works
Self-citationWei, C.-H., Peng, Y., Leaman, R., Davis, A. P., Mattingly, C. J., Li, J., Wiegers, T. C., & Lu, Z. (2016) · Database · doi:10.1093/database/baw032 · Cited by 198 · see citing works
Self-citationDrug–drug interaction extraction via hierarchical RNNs on sequence and shortest dependency paths
Zhang, Y., Zheng, W., Lin, H., Wang, J., Yang, Z., & Dumontier, M. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx659 · Cited by 192 · see citing works
Challenges in clinical natural language processing for automated disorder normalization
Leaman, R., Khare, R., & Lu, Z. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.07.010 · Cited by 188 · see citing works
Self-citationSciBERT: Pretrained Contextualized Embeddings for Scientific Text
Beltagy, I., Cohan, A., & Lo, K. (2019) · Cited by 173 · see citing works
Automatic extraction of relations between medical concepts in clinical texts
Rink, B., Harabagiu, S. M., & Roberts, K. E. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000153 · Cited by 172 · see citing works
Automatic semantic classification of scientific literature according to the hallmarks of cancer
Baker, S., Silins, I., Guo, Y., Ali, I., Högberg, J., Stenius, U., & Korhonen, A. (2016) · Bioinform. · doi:10.1093/bioinformatics/btv585 · Cited by 166 · see citing works
BIOSSES: a semantic sentence similarity estimation system for the biomedical domain
Soğancıoğlu, G., Öztürk, H., & Özgür, A. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx238 · Cited by 159 · see citing works
ML-Net: multi-label classification of biomedical texts with deep neural networks
Du, J., Chen, Q.-Y., Peng, Y., Xiang, Y., Tao, C., & Lu, Z.-Y. (2018) · J. Am. Medical Informatics Assoc. · doi:10.1093/jamia/ocz085 · Cited by 148 · see citing works
Self-citationMedSTS: a resource for clinical semantic textual similarity
Wang, Y., Afzal, N., Fu, S., Wang, L., Shen, F., Rastegar-Mojarad, M., & Liu, H. (2020) · Language Resources and Evaluation · doi:10.1007/s10579-018-9431-1 · Cited by 124 · see citing works
Extracting chemical–protein relations with ensembles of SVM and deep learning models
Peng, Y., Rios, A., Kavuluru, R., & Lu, Z. (2018) · Database · doi:10.1093/database/bay073 · Cited by 123 · see citing works
Self-citationProbing Biomedical Embeddings from Language Models
Jin, Q., Dhingra, B., Cohen, W., & Lu, X. (2019) · doi:10.18653/v1/w19-2011 · Cited by 123 · see citing works
The Role of Context Types and Dimensionality in Learning Word Embeddings
Melamud, O., McClosky, D., Patwardhan, S., & Bansal, M. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1118 · Cited by 107 · see citing works
How user intelligence is improving PubMed
Fiorini, N., Leaman, R., Lipman, D. J., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4267 · Cited by 79 · see citing works
Probing Biomedical Embeddings from Language ModelsPreprint
Jin, Q., Dhingra, B., Cohen, W. W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02181 · Cited by 29 · see citing works
Smalheiser, N. R., Cohen, A. M., & Bonifield, G. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103096 · Cited by 23 · see citing works
Elucidating high-dimensional cancer hallmark annotation via enriched ontology
Yan, S., & Wong, K. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.07.011 · Cited by 6 · see citing works
Self-citation(2015) · doi:10.4230/oasics.slate.2021.8 · Cited by 1 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Research works printed but not linked: 4
- Qingyu Chen, Yifan Peng, and Zhiyong Lu. 2019. BioSentVec: creating sentence embeddings for biomedical texts. In Proceedings of the 7th IEEE In- ternational Conference on Healthcare Informatics.
- Martin Krallinger, Obdulia Rabal, Saber A. Akhondi, Martín Pérez Pérez, Jesús Santamaría, Gael Pérez Rodríguez, Georgios Tsatsaronis, Ander Intxau- rrondo, José Antonio López1 Umesh Nandal, Erin Van Buel, Akileshwari Chandrasekhar, Mar- leen Rodenburg, Astrid Laegreid, Marius Doornen- bal, Julen Oyarzabal, Analia Lourenc ¸o, and Alfonso Valencia. 2017. Overview of the BioCreative VI chemical-protein interaction track. In Proceedings of BioCreative, pages 141-146.
- Yuqi Si, Jingqi Wang, Hua Xu, and Kirk Roberts. 2019. Enhancing clinical concept extraction with contex- tual embedding. arXiv preprint arXiv:1902.08691. · arXiv
- Wonjin Yoon, Chan Ho So, Jinhyuk Lee, and Jaewoo Kang. 2018. CollaboNet: collaboration of deep neu- ral networks for biomedical named entity recogni- tion. arXiv preprint arXiv:1809.07950. · arXiv
Cited by
Sorted by citations
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170 · see citing works
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150 · see citing works
ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model
Huang, H., Zheng, O., Wang, D., Yin, J., Wang, Z., Ding, S., Yin, H., Xu, C., Yang, R., Zheng, Q., et al. (2023) · International Journal of Oral Science · doi:10.1038/s41368-023-00239-y · Cited by 324 · see citing works
Smit, A., Jain, S., Rajpurkar, P., Pareek, A., Ng, A., & Lungren, M. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.117 · Cited by 262 · see citing works
A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis
Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., & Pei, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11030219 · Cited by 242 · see citing works
Chen, Q., Hu, Y., Peng, X., Xie, Q., Jin, Q., Gilson, A., Singer, M. B., Ai, X., Lai, P.-T., Wang, Z., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-56989-2 · Cited by 219 · see citing works
“The remarkable advancements have sparked extensive discussions among society, with excitement and concerns alike.”
Self-citationLimitations of transformers on clinical text classification
Gao, S., Alawad, M., Young, M. T., Gounley, J., Schaefferkoetter, N., Yoon, H. J., Wu, X.-C., Durbin, E. B., Doherty, J., Stroup, A., et al. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3062322 · Cited by 157 · see citing works
A comprehensive survey on relation extraction: recent advances and new frontiers
Zhao, X., Deng, Y., Yang, M., Wang, L., Zhang, R., Cheng, H., Lam, W., Shen, Y., & Xu, R. (2024) · ACM Computing Surveys · doi:10.1145/3674501 · Cited by 168 · see citing works
Improving chest X-ray report generation by leveraging warm starting
Nicolson, A., Dowling, J., & Koopman, B. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102633 · Cited by 161 · see citing works
A medical multimodal large language model for future pandemics
Liu, F., Zhu, T., Wu, X., Yang, B., You, C., Wang, C., Lu, L., Liu, Z., Zheng, Y., Sun, X., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00952-2 · Cited by 155 · see citing works
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102963 · Cited by 147 · see citing works
Recent advances in biomedical literature mining
Zhao, S., Su, C., Lu, Z., & Wang, F. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa057 · Cited by 134 · see citing works
Self-citationLexGLUE: A Benchmark Dataset for Legal Language Understanding in English
Chalkidis, I., Jana, A., Hartung, D., Bommarito, M. J., Androutsopoulos, I., Katz, D. M., & Aletras, N. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3936759 · Cited by 111 · see citing works
Smit, A., Jain, S., Rajpurkar, P., Pareek, A., Ng, A. Y., & Lungren, M. P. (2020) · arXiv · doi:10.48550/arxiv.2004.09167 · Cited by 103 · see citing works
Biomedical Named Entity Recognition with Multilingual BERT
Hakala, K., & Pyysalo, S. (2019) · Proceedings of The 5th Workshop on BioNLP Open Shared Tasks · doi:10.18653/v1/d19-5709 · Cited by 96 · see citing works
Fraile Navarro, D., Ijaz, K., Rezazadegan, D., Rahimi-Ardabili, H., Dras, M., Coiera, E., & Berkovsky, S. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105122 · Cited by 82 · see citing works
Mao, C., Xu, J., Rasmussen, L., Li, Y., Adekkanattu, P., Pacheco, J., Bonakdarpour, B., Vassar, R., Shen, L., Jiang, G., et al. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104442 · Cited by 67 · see citing works
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2024) · doi:10.2139/ssrn.4809363 · Cited by 69 · see citing works
RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
Jain, S., Agrawal, A., Saporta, A., Truong, S. Q., Duong, D. N., Bui, T., Chambon, P., Zhang, Y., Lungren, M. P., Ng, A. Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.14463 · Cited by 66 · see citing works
“Biomedical pretraining For each of our approaches, in addition to using BERT weight initial-izations, we use weight initializations from four different biomedical pretrained models, which are BioBERT [35], Bio+ClinicalBERT [36], PubMedBERT [37], and BlueBERT [38].”
MethodA pre-training and self-training approach for biomedical named entity recognition
Gao, S., Kotevska, O., Sorokine, A., & Christian, J. B. (2021) · PLOS ONE · doi:10.1371/journal.pone.0246310 · Cited by 46 · see citing works
Wu, J., Liu, X., Li, M., Li, W., Su, Z., Lin, S., Garay, L., Zhang, Z., Zhang, Y., Zeng, Q., et al. (2024) · NEJM AI · doi:10.1056/aira2400012 · Cited by 41 · see citing works
Yang, X., Zhang, H., He, X., Bian, J., & Wu, Y. (2020) · JMIR Medical Informatics · doi:10.2196/22982 · Cited by 29 · see citing works
Knowledge Graph-Enhanced Large Language Models via Path Selection
Liu, H., Wang, S., Zhu, Y., Dong, Y., & Li, J. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.376 · Cited by 28
Assertion Detection in Clinical Notes: Medical Language Models to the Rescue?
van Aken, B., Trajanovska, I., Siu, A., Mayrdorfer, M., Budde, K., & Loeser, A. (2021) · Proceedings of the Second Workshop on Natural Language Processing for Medical Conversations · doi:10.18653/v1/2021.nlpmc-1.5 · Cited by 25 · see citing works
Al-Hussaini, I., Nakajima An, D., Lee, A. J., Bi, S., & Mitchell, C. S. (2022) · 2022 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata55660.2022.10020807 · Cited by 10 · see citing works
de Arriba-Pérez, F., & García-Méndez, S. (2025) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-024-09508-2 · Cited by 8 · see citing works
Lee, H. G., Sholle, E., Beecy, A., Al’Aref, S., & Peng, Y. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.358 · Cited by 7 · see citing works
Self-citationAlgorithmic bias, generalist models, and clinical medicine
Keeling, G. (2024) · AI and Ethics · doi:10.1007/s43681-023-00329-x · Cited by 5 · see citing works
Abdaoui, H., Barki, C., Dergaa, I., Tlili, K., Ceylan, H. İ., Bragazzi, N. L., de Giorgio, A., Ben Salah, R., & Rahmouni, H. B. (2025) · Bioengineering · doi:10.3390/bioengineering13010030 · Cited by 6 · see citing works
- Authors
- Peng, Y., Yan, S., & Lu, Z.
- Affiliations
- National Institutes of Health (US)
- Venue
- Proceedings of the 18th BioNLP Workshop and Shared Task
- Published
- 2019
- Pages
- 58-65
- OpenAlex
- W2971258845
- Cited by
- 899
- References
- 44
- Citations by year
- 2026: 66 · 2025: 136 · 2024: 142 · 2023: 164 · 2022: 147 · 2021: 150 · 2020: 83 · 2019: 11
- FWCI
- 37.41
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- BioNLP Workshop and Shared Task (BioNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Institutes of Health
- U.S. National Library of Medicine (R00LM013001, K99LM013001)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Keywords
- transfer learning, biomedical natural language processing, BERT, ELMo, biomedical text understanding, clinical text understanding, PubMed abstracts, benchmark dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
