SciBERT: A Pretrained Language Model for Scientific Text
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index; 9 of the 33 sources its claims cite were checked for support.
Abstract
Iz Beltagy, Kyle Lo, Arman Cohan. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
CoChat Summary
TL;DR
SciBERT is a BERT-based pretrained language model trained on a 3.17B-token corpus of 1.14M scientific papers, with an in-domain WordPiece vocabulary. It improves over BERT-Base across scientific NLP tasks and achieves new state-of-the-art results on several datasets.
Key findings
- SciBERT outperforms BERT-Base on scientific tasks by +2.11 F1 with finetuning and +2.43 F1 without finetuning.
- Domain breakdowns: biomedical +1.92 F1 with finetuning and +3.59 F1 without; computer science +3.55 F1 with finetuning and +1.13 F1 without; multidomain +0.49 F1 with finetuning and +0.93 F1 without.
- Finetuning beats frozen BERT embeddings by +3.25 F1 with SciBERT and +3.58 F1 with BERT-Base on average.
- Using SCIVOCAB instead of BASEVOCAB gives +0.60 F1 averaged across datasets: +0.76 F1 biomedical, +0.61 F1 computer science, +0.11 F1 multidomain. SciBERT achieves new SOTA on BC5CDR, ChemProt, EBM-NLP, ACL-ARC, the NER part of SciERC, and SciCite.
Methods
- SciBERT follows the BERT-Base architecture and is pretrained on a random sample of 1.14M Semantic Scholar papers: 18% computer science and 82% broad biomedical, using full text, averaging 154 sentences (2,769 tokens) per paper, for 3.17B tokens.
- The authors build SCIVOCAB, a 30K WordPiece vocabulary with 42% token overlap with BASEVOCAB, and train four variants: cased/uncased and BASEVOCAB/SCIVOCAB.
- Evaluation covers NER, PICO extraction, text classification, relation classification, and dependency parsing, using datasets including EBM-NLP, SciERC, ACL-ARC, SciCite, and Paper Field; results are macro F1 for most tasks, micro F1 for ChemProt, and LAS/UAS for DEP, averaged over multiple random seeds.
Limitations
- The paper states future work will release a BERT-Large analogue and experiment with different proportions of papers from each domain, noting these models are costly to train and aiming for a single resource useful across multiple domains.
- The paper notes SciBERT performs slightly worse than SOTA on JNLPBA, NCBI-disease, and GENIA, attributing these to comparison models: a BiLSTM-CRF ensemble trained on multiple NER datasets, BIOBERT trained on 18B biomedical tokens, and a parser using POS features not used here.
- Evident from the text: the pretraining corpus is limited to computer science and biomedical papers, so coverage of other scientific domains is not established; frozen-embedding experiments did not use extensive hyperparameter search; and SciERC relation results are not comparable to prior work because this paper uses gold entities while prior work performs joint entity and relation extraction.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
9 Valid · 0 Not valid · 16 Not checked · 29 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Yet while both BERT and ELMo have released pretrained models, they are still trained on general domain corpora such as news articles and Wikipedia.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites Devlin et al., 2019 matched to “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” (2019) source
- Partly supportedcites Peters et al., 2018 matched to “Deep Contextualized Word Representations” (2018) sourceExperimental — accuracy not yet measured
Section 3.4 says the ELMo biLM was pretrained on the 1B Word Benchmark, and the Introduction says the trained models were made publicly available. The paper does not discuss BERT or Wikipedia, or identify the benchmark text as news articles, so it does not support those parts of the sentence. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“As shown through ELMo (Peters et al., 2018), GPT (Radford et al., 2018) and BERT (Devlin et al., 2019), unsupervised pretraining of language models on large corpora significantly improves performance on many NLP tasks.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 1 Introduction
What the sources say
- Supportedcites Peters et al., 2018 matched to “Deep Contextualized Word Representations” (2018) source
Section 2 describes pretraining ELMo’s bidirectional language model on unlabeled data, and Section 3.4 identifies the large 1B Word Benchmark training corpus. Section 4 reports that adding ELMo improved results across six NLP benchmark tasks, with relative error reductions of 6–20%. This supports the claim as it relates to ELMo; the other cited models are not attributed to this source. (LLM, conf=0.92)
- Not matchedcites Radford et al., 2018
An author–year citation the paper’s reference list doesn’t settle. We don’t search these by name.
- Not checked — no full text of the sourcecites Devlin et al., 2019 matched to “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” (2019) source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“SCIBERT significantly outperformed BERT-Base and achieves new SOTA results on several of these tasks, even compared to some reported BIOBERT (Lee et al., 2019) results on biomedical tasks.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 7 Conclusion and Future Work
What the sources say
- Supportedcites Lee et al., 2019 matched to “BioBERT: a pre-trained biomedical language representation model for biomedical text mining” (2019) source
The citation identifies the source of the reported BioBERT results used as a comparator, rather than attributing the SciBERT comparison to Lee et al. The Experimental results (§4.3) report BioBERT performance on biomedical named-entity recognition, relation extraction, and question-answering tasks. (LLM, conf=0.90)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In Table 2, we compare SCIBERT results with reported BIOBERT results on the subset of datasets included in (Lee et al., 2019) BC5CDR and ChemProt, and performs similarly on JNLPBA despite being trained on a substantially smaller biomedical corpus.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 4.1 Biomedical Domain
What the sources say
- Supportedcites Lee et al., 2019 matched to “BioBERT: a pre-trained biomedical language representation model for biomedical text mining” (2019) source
The results tables in our copy report BioBERT scores for BC5CDR, ChemProt, and JNLPBA (section 6; page unavailable). The citation identifies the source of those reported BioBERT results; the comparison with SciBERT is the citing authors’ own conclusion. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In addition, SCIBERT outperforms the SOTA on SciCite (Co-han et al., 2019).”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 4.3 Multiple Domains
What the sources say
- Partly supportedcites Cohan et al., 2019 matched to “Structural Scaffolds for Citation Intent Classification in Scientific Publications” (2019) sourceExperimental — accuracy not yet measured
Table 4 (section 4.3 Results; page unavailable) shows that the paper’s BiLSTM-Attn model with ELMo and both scaffolds outperformed the listed previous SOTA on SciCite (84.0 versus 79.6 macro F1). It does not report a SciBERT result, so it does not support attributing that improvement to SciBERT. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
17 more claims
Create a free account to see all 54 claims and to verify the 13 references still open.
- Probably not a substantial claim. 17
- It may state two separate claims as one. 17
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“With their word-piece vocabulary, contextual embeddings, and self-supervised pre-training on large unlabelled corpora these models have essentially removed the need for most pre-processing beyond automatically-applied lowercasing. 14,31”
From Data extraction methods for systematic review (semi)automation: Update of a living systematic review (2025) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“SciBERT, introduced by Beltagy et al. [16], was the first language model tailored for scientific texts.”
From Evaluating Large Language Models for Structured Science Summarization in the Open Research Knowledge Graph (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Hence, we base our experiments on well-established large language models (LLM) from the general and scientific domain -RoBERTa (Liu et al., 2019), BioBERT (Lee et al., 2020) and SciBERT ( Beltagy et al., 2019) -with the recommended default hyperparameter settings.”
From NLPeer: A Unified Resource for the Computational Study of Peer Review (2023) · cites this paper as Beltagy et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Although the first experiments with BERT ( 39 ) were disappointing, fine-tuning SciBERT ( 40 ) led to the best results and is the current production architecture of the platform.”
From scite: a smart citation index that displays the context of citations and classifies their intent using deep learning (2021) · cites this paper as [40]
The cited source says what the claim attributes to it.
In the citing paper
“Similar observations have been made in prior work, e.g. Beltagy et al. (2019) found that pre-training BERT on text from research papers improved its performance on scientific tasks.”
From Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019) · cites this paper as Beltagy et al. (2019)
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 7×
“3617 Casing We follow Devlin et al. (2019) in using the cased models for NER and the uncased models for all other tasks.”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 5×
“Recent work on domain adaptation of BERT includes B IO B ERT (Lee et al., 2019) and C LINI - CAL B ERT (Alsentzer et al., 2019; Huang et al., 2019).”
MethodResultInfluentialGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · cited 2×
“We finetune for 2 to 5 epochs using a batch size of 32 and a learning rate of 5e-6, 1e5, 2e-5, or 5e-5 with a slanted triangular schedule (Howard and Ruder, 2018) which is equivalent to the linear warmup followed by linear decay (Devlin et al.”
Method
Cited by
Items 201–300 of 3,596
CySecBERT : A Domain-Adapted Language Model for the Cybersecurity Domain
Bayer, M., Kuehn, P., Shanehsaz, R., & Reuter, C. (2024) · ACM Transactions on Privacy and Security · doi:10.1145/3652594 · Cited by 101
“BioBERT [18] biomedical PT (+ FT) BERT SciBERT [5] scientific PT (+ FT) BERT [11] Papers (bio.”
BackgroundMethodInfluentialLearning multi-modal representations by watching hundreds of surgical video lectures
Yuan, K., Srivastav, V., Yu, T., Lavanchy, J. L., Marescaux, J., Mascagni, P., Navab, N., & Padoy, N. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103644 · Cited by 122
AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization
Yu, T., Liu, Z., & Fung, P. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.471 · Cited by 116
Context Matters: A Strategy to Pre-train Language Model for Science Education
Liu, Z., He, X., Liu, L., Liu, T., & Zhaı, X. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-36336-8_103 · Cited by 93
“In Figure 3, we can see that SciBERT [3] is a science-oriented version of BERT developed through in-domain pre-training.”
BackgroundMethodInfluentialGPT-3 Models are Poor Few-Shot Learners in the Biomedical DomainPreprint
Moradi, M., Blagec, K., Haberl, F., & Samwald, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.02555 · Cited by 95
“Domain-specific language models such as BioBERT (Lee et al., 2019), SciBERT (Beltagy et al., 2019), and Clinical XLNet (Huang et al., 2020) were pretrained on very large biomedical and clinical text corpora; they achieved state-of-the-art results after task-specific fine-tuning on (bio)medical datasets.”
Weakly Supervised Sequence Tagging from Noisy Rules
Safranchik, E., Luo, S., & Bach, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6009 · Cited by 78
A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extraction
Shen, Y., Ma, X., Tang, Y., & Lu, W. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449895 · Cited by 67
“On the SciERC dataset, we replace BERT with SciBERT [6].”
BioM-Transformers: Building Large Biomedical Language Models with BERT, ALBERT and ELECTRA
Alrowili, S., & Shanker, V. K. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.24 · Cited by 117
Technology identification from patent texts: A novel named entity recognition method
Puccetti, G., Giordano, V., Spada, I., Chiarello, F., & Fantoni, G. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.122160 · Cited by 91
Paragraph-level Simplification of Medical Texts
Devaraj, A., Marshall, I., Wallace, B., & Li, J. J. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.395 · Cited by 79
Electric Power Audit Text Classification With Multi-Grained Pre-Trained Language Model
Meng, Q., Song, Y., Mu, J., Lv, Y., Yang, J., Xu, L., Zhao, J., Ma, J., Yao, W., Wang, R., et al. (2023) · IEEE Access · doi:10.1109/access.2023.3240162 · Cited by 83
Hunting Conspiracy Theories During the COVID-19 Pandemic
Moffitt, J. D., King, C., & Carley, K. M. (2021) · Social Media + Society · doi:10.1177/20563051211043212 · Cited by 99
“These applications include BioBERT (biomedical text (Lee et al., 2020)), SciBERT (scientific applications (Beltagy et al., 2019)), DocBERT (document classification (Devlin et al., 2018)), and COVID-Twitter-BERT (Müller et al., 2020).”
BackgroundMethodApplication of Transformers in Cheminformatics
Luong, K.-D., & Singh, A. K. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c02070 · Cited by 85
Scholarly knowledge graphs through structuring scholarly communication: a review
Verma, S., Bhatia, R., Harit, S., & Batish, S. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00806-6 · Cited by 78
“SciBERT [49] performed extensive experimentation on multi-domain corpus.”
BackgroundUribe, S. E., Maldupa, I., Kavadella, A., El Tantawi, M., Chaurasia, A., Fontana, M., Marino, R., Innes, N., & Schwendicke, F. (2024) · European Journal of Dental Education · doi:10.1111/eje.13009 · Cited by 77
“49 The development of SciEdBERT, 50 an education-focused language model, suggests the potential for domain-specific LLMs in dental education.”
BackgroundFast changeset-based bug localization with BERT
Ciborowska, A., & Damevski, K. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510042 · Cited by 60
“Researchers in the biomedical domain recognized the same issue affecting medical terms and proposed domain-specific BERT adaptations [2, 11, 26].”
BackgroundLiu, Y., Zhang, X. F., Wegsman, D., Beauchamp, N., & Wang, L. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.101 · Cited by 115
PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation
Liu, Y., Sharma, P., Oswal, M., Xia, H., & Huang, Y. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735789 · Cited by 87
“During the query decomposition step, we prompt the LLM to first break down the user’s initial research question into primary queries that capture various aspects of the main topic.”
Frisoni, G., Moro, G., & Carbonaro, A. (2021) · IEEE Access · doi:10.1109/access.2021.3130956 · Cited by 84
DFFNDDS: prediction of synergistic drug combinations with dual feature fusion networks
Xu, M., Zhao, X., Wang, J., Feng, W. Q., Wen, N., Wang, C., Wang, J., Liu, Y., & Zhao, L. (2023) · Journal of Cheminformatics · doi:10.1186/s13321-023-00690-3 · Cited by 82
“For instance, ChemBERT [31], DeepChem [32], and SciBERT [33] were developed to apply deep learning in drug discovery.”
BackgroundWhen FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain
Shah, R., Chawla, K., Eidnani, D., Shah, A., Du, W., Chava, S., Raman, N., Smiley, C., Chen, J., & Yang, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.148 · Cited by 74
Deep Graph Learning for Anomalous Citation Detection
Liu, J., Xia, F., Xu, F., Ren, J. F., & Liu, H. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3145092 · Cited by 87
Arslan, Y., Allix, K., Veiber, L., Lothritz, C., Bissyandé, T. F., Klein, J., & Goujon, A. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442442.3451375 · Cited by 98
“Although SciBERT was trained on Biomedical and Computer Science papers from scratch, we note that its custom vocabulary has only a 42% overlap with the BERT vocabulary, suggesting that the custom vocabulary contributed to the performance improvement.”
BackgroundMethodInfluentialBeyond word embeddings: A survey
Incitti, F., Urli, F., & Snidaro, L. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.08.024 · Cited by 97
Mohajer-Bastami, A., Moin, S., Ahmad, S., Ahmed, A., Pouwels, S., Hajibandeh, S., Yang, W., Parmar, C. D., Kermansaravi, M., Khalil, M., et al. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1644041 · Cited by 86
Joint Biomedical Entity and Relation Extraction with Knowledge-Enhanced Collective Inference
Lai, T., Ji, H., Zhai, C., & Tran, Q. H. (2021) · doi:10.18653/v1/2021.acl-long.488 · Cited by 84
“Our model first constructs a contextualized representation for each input token using SciBERT (Beltagy et al., 2019).”
BackgroundMethodInfluentialRapidly Bootstrapping a Question Answering Dataset for COVID-19Preprint
Tang, R., Nogueira, R. F., Zhang, E., Gupta, N., Cam, P., Cho, K., & Lin, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.11339 · Cited by 82
Exploration of crystal chemical space using text-guided generative artificial intelligence
Park, H., Onwuli, A., & Walsh, A. (2025) · Nature Communications · doi:10.1038/s41467-025-59636-y · Cited by 104
Evaluating GPT and BERT models for protein–protein interaction identification in biomedical text
Rehana, H., Çam, N. B., Basmaci, M., Zheng, J., Jemiyo, C., He, Y. O., Özgür, A., & Hur, J. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae133 · Cited by 78
“5 and GPT-4 variants, against masked language models like BioBERT, PubMedBERT, and SciBERT in a PPI prediction task.”
MethodInfluentialBenchmarking for biomedical natural language processing tasks with a domain specific ALBERT
Naseem, U., Dunn, A. G., Khushi, M., & Kim, J. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04688-w · Cited by 67
“Datasets BioBERT [13] SciBERT[11] BLUE [12] PubMedBERT [14] KeBioLM [15] BioALBERT”
MethodInfluentialEvidence-based Fact-Checking of Health-related Claims
Sarrouti, M., Ben Abacha, A., Mrabet, Y., & Demner-Fushman, D. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.297 · Cited by 71
Houssein, E. H., Mohamed, R. E., & Ali, A. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-34294-6 · Cited by 83
Hadi, M. U., tashi, Q. A., Qureshi, R., Shah, A., muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., et al. (2023) · doi:10.36227/techrxiv.23589741.v3 · Cited by 53
Zhang, Y., Sui, X., Pan, F., Yu, K., Li, K., Tian, S., Erdengasileng, A., Han, Q., Wang, W., Wang, J., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01014-w · Cited by 103
“Recently, the technique of pretraining and large language models (LLMs) have garnered considerable attention, expanding both model complexity and the amount of training data and achieving remarkable progress in information retrieval tasks 43–54 .”
A Comparative Analysis of Active Learning for Biomedical Text Mining
Naseem, U., Khushi, M., Khan, S. K., Shaukat, K., & Moni, M. A. (2021) · Applied System Innovation · doi:10.3390/asi4010023 · Cited by 82
“Peng et al. [65] introduced Bio-204 medical Language Understanding Evaluation (BLUE), a collection of resources for evalu-205 ating and analysing natural biomedical language representation models.”
BackgroundPre-training Methods in Information Retrieval
Fan, Y., Xie, X., Cai, Y., Chen, J., Ma, X., Li, X., Zhang, R., & Guo, J. (2022) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000100 · Cited by 98
Han, B., Susnjak, T., & Mathrani, A. (2024) · Applied Sciences · doi:10.3390/app14199103 · Cited by 68
Biomedical Named Entity Recognition at Scale
Kocaman, V., & Talby, D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68763-2_48 · Cited by 96
Makridis, G., Mavrepis, P., & Kyriazis, D. (2022) · Machine Learning · doi:10.1007/s10994-022-06151-6 · Cited by 68
“A well-known paradigm of such language model achieving impressive results is SciBERT, a Bidirectional Encoder Representations from Transformers (Devlin et al. 2018) (BERT) based model leveraging unsupervised pre-training on a large multi-domain corpus (Beltagy et al. 2019).”
MethodDeep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research
Gruetzemacher, R., & Paradice, D. (2022) · ACM Computing Surveys · doi:10.1145/3505245 · Cited by 95
Artificial intelligence (AI) in drug design and discovery: A comprehensive review
Bhat, A. R., & Ahmed, S. (2025) · In Silico Research in Biomedicine · doi:10.1016/j.insi.2025.100049 · Cited by 78
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations
Singh, A., D’Arcy, M., Cohan, A., Downey, D., & Feldman, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.338 · Cited by 67
Self-citationA pre-trained BERT for Korean medical natural language processing
Kim, Y., Kim, J.-H., Lee, J. M., Jang, M. J., Yum, Y., Kim, S., Shin, U., Kim, Y.-M., Joo, H. J., & Song, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-17806-8 · Cited by 77
Porciello, J., Ivanina, M., Islam, M., Einarson, S., & Hirsh, H. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00235-5 · Cited by 71
Constructing and analyzing domain-specific language model for financial text mining
Suzuki, M., Sakaji, H., Hirano, M., & Izumi, K. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103194 · Cited by 80
Identifying Machine-Paraphrased Plagiarism
Wahle, J. P., Ruas, T., Foltýnek, T., Meuschke, N., & Gipp, B. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-96957-8_34 · Cited by 37
“Recently, the NLP community adapted and extended the neural language model BERT [8] for a variety of tasks [2, 5, 33, 34, 45, 49, 54], similar to the way that word2vec [31] has influenced many later models in NLP [4, 41, 42].”
MethodPeer review analyze: A novel benchmark resource for computational analysis of peer reviews
Ghosal, T., Kumar, S., Bharti, P. K., & Ekbal, A. (2022) · PLOS ONE · doi:10.1371/journal.pone.0259238 · Cited by 66
“The BERT baselines are pre-trained on large scientific domain data (using SciBERT [61] pre-trained weights) and also use the context to predict the sentence labels.”
MOOC-BERT: Automatically Identifying Learner Cognitive Presence From MOOC Discussion Data
Liu, Z., Kong, X., Chen, H., Liu, S., & Yang, Z. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3240715 · Cited by 71
“Pretrained language models (PLMs), especially bidirectional encoder representations from transformers (BERT), have become prevalent and helped achieve significant performance gains in natural language processing (NLP) tasks [25], [26], [27].”
BackgroundInfluentialSteel design based on a large language model
Tian, S., Jiang, X. F., Wang, W., Jing, Z., Zhang, C., Zhang, C., Lookman, T., & Su, Y. (2024) · Acta Materialia · doi:10.1016/j.actamat.2024.120663 · Cited by 74
Dessì, D., Osborne, F., Recupero, D. R., Buscaldi, D., & Motta, E. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109945 · Cited by 79
“This module uses the scibert scivocab uncased [30] transformer model which is pre-trained on the full texts of 1.14M papers from Semantic Scholar 29 .”
BackgroundMethodRöttger, P., & Pierrehumbert, J. B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.206 · Cited by 102
TTPXHunter: Actionable Threat Intelligence Extraction as TTPs from Finished Cyber Threat Reports
Rani, N., Saha, B., Maurya, V., & Shukla, S. K. (2024) · Digital Threats Research and Practice · doi:10.1145/3696427 · Cited by 82
“This section presents the details of every framework, tool, and method used to get insight into their background, which is required to understand the methodology.”
MethodPretrained Biomedical Language Models for Clinical NLP in Spanish
Carrino, C. P., Palao, J. L., Pàmies, M., Gutiérrez-Fandiño, A., Armengol-Estapé, J., Silveira-Ocampo, J., Valencia, A., González-Agirre, A., & Villegas, M. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.19 · Cited by 99
Rezayi, S., Liu, Z., Wu, Z., Dhakal, C., Ge, B., Dai, H., Mai, G., Liu, N., Zhen, C., Liu, T., et al. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2024.3442542 · Cited by 79
Data-Centric Foundation Models in Computational Healthcare: A Survey
Zhang, Y., Gao, J. G., Tan, Z., Zhou, L., Ding, K., Zhou, M., Zhang, S., & Wang, D. (2026) · ACM Computing Surveys · doi:10.1145/3800677 · Cited by 74
“For instance, SciBERT [10] and PubMedBERT [79] are pre-trained on multi-domain scientific publications and biomedical domain-specific corpora, respectively, based on the BERT strategy.”
ClinicalT5: A Generative Language Model for Clinical Text
Lu, Q., Dou, D., & Nguyen, T. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.398 · Cited by 64
A clinical specific BERT developed using a huge Japanese clinical text corpus
Kawazoe, Y., Shibata, D., Shinohara, E., Aramaki, E., & Ohe, K. (2021) · PLoS ONE · doi:10.1371/journal.pone.0259763 · Cited by 83
“In the fields of life science and clinical medicine, domain-specific pre-trained models, such as Sci-BERT [9], Bio-BERT [10], and Clinical-BERT [11], have been published for English texts.”
BackgroundCitation Intent Classification Using Word Embedding
Roman, M., Shahid, A., Khan, S., Koubaa, A., & Yu, L. (2021) · IEEE Access · doi:10.1109/access.2021.3050547 · Cited by 57
Joint Modeling of Chest Radiographs and Radiology Reports for Pulmonary Edema Assessment
Chauhan, G., Liao, R., Wells, W. M., Andreas, J., Wang, X., Berkowitz, S. J., Horng, S., Szolovits, P., & Golland, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59713-9_51 · Cited by 89
A Comparative Study of Using Pre-trained Language Models for Toxic Comment Classification
Zhao, Z., Zhang, Z., & Hopfgartner, F. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442442.3452313 · Cited by 80
“SciBERT [3] and BioBERT [24] are two examples of łcontinued pre-training in domainž, which both fine-tuned BERT using large domain corpora and require significant computing resources and time [18].”
BackgroundMethodInfluentialNiksirat, K. S., Goswami, L., Rao, P. S. B., Tyler, J., Silacci, A., Aliyu, S., Aebli, A., Wacharamanotham, C., & Cherubini, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580848 · Cited by 74
“Further, the availability of labeled data and using a model trained only on scientifc articles like SciBERT [13] might also improve the results.”
MethodThe Semantic Scholar Open Data Platform
Kinney, R., Anastasiades, C., Authur, R., Beltagy, I., Bragg, J., Buraczynski, A., Cachola, I., Candra, S., Chandrasekhar, Y., Cohan, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.10140 · Cited by 64
Self-citationAspect-based Sentiment Analysis of Scientific Reviews
Chakraborty, S., Goyal, P., & Mukherjee, A. (2020) · Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 · doi:10.1145/3383583.3398541 · Cited by 63
“Next we use a bunch of modern deep learning techniques like BiLSTM with CNN, FFNN fed with (i) Google’s universal sentence encoder embeddings (FFNN-uni) [5] and (ii) SciBERT embeddings (FFNN-sci) [4].”
MethodTWilBert: Pre-trained deep bidirectional transformers for Spanish Twitter
González, J. Á., Hurtado, L.-F., & Plà, F. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.09.078 · Cited by 69
Characteristics of the 100 largest modern zoonotic disease outbreaks
Stephens, P. R., Gottdenker, N. L., Schatz, A. M., Schmidt, J. P., & Drake, J. M. (2021) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2020.0535 · Cited by 72
“[117,118]) to generate more detailed and complete databases of outbreak characteristics represents an exciting avenue for future work.”
BackgroundZhang, C., Tian, L., & Chu, H. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103507 · Cited by 76
TweetBERT: A Pretrained Language Representation Model for Twitter Text AnalysisPreprint
Qudar, M. M. A., & Mago, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11091 · Cited by 65
Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies
Srinath, M., Wilson, S., & Giles, C. L. (2021) · doi:10.18653/v1/2021.acl-long.532 · Cited by 95
Using the contextual language model BERT for multi-criteria classification of scientific articles
Ambalavanan, A. K., & Devarakonda, M. V. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103578 · Cited by 77
Survey of BERT (Bidirectional Encoder Representation Transformer) types
Mohammed, A. H., & Ali, A. H. (2021) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1963/1/012173 · Cited by 83
“REL special case of classification of text, so the model able to predict type of relation between entities [21][22][23][24].”
BackgroundDomain-Independent Extraction of Scientific Concepts from Research Articles
Brack, A., D’Souza, J., Hoppe, A., Auer, S., & Ewerth, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-45439-5_17 · Cited by 75
Lin, J., Yu, Y., Zhou, Y., Zhou, Z., & Shi, X. (2020) · Scientometrics · doi:10.1007/s11192-020-03430-8 · Cited by 51
NER-BERT: A Pre-trained Model for Low-Resource Entity TaggingPreprint
Liu, Z., Jiang, F., Hu, Y., Chen, S., & Fung, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.00405 · Cited by 78
“…modeling and down-stream tasks, task-specific pre-training methods have been proposed to further boost the task performance, such as SciBERT (Beltagy et al., 2019), VideoBERT (Sun et al., 2019), DialoGPT (Zhang et al., 2020), PLATO (Bao et al., 2020), Code-BERT (Feng et al., 2020), ToD-BERT…”
MethodInfluentialZhang, Y., & Zhang, H. (2023) · Neural Processing Letters · doi:10.1007/s11063-023-11266-5 · Cited by 74
“The aim of MRC models [16–19 , 35 ] is to extract the answer spans from a context through a query.”
Improving Log-Based Anomaly Detection by Pre-Training Hierarchical Transformers
Huang, S.-H., Liu, Y., Fung, C. J., Wang, H., Yang, H.-L., & Luan, Z.-Z. (2023) · IEEE transactions on computers · doi:10.1109/tc.2023.3257518 · Cited by 54
“SciBERT [21] build domain-specific vocabulary and pre-train”
Biomedical Event Extraction with Hierarchical Knowledge Graphs
Huang, K.-H., Yang, M., & Peng, N. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.114 · Cited by 92
Hadi, M. A., & Fard, F. H. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10314-x · Cited by 71
“PTMs that are trained on general purpose data) for NLP tasks in fields such as science or law [21, 22].”
BackgroundKhatua, A., Mailthody, V. S., Taleka, B., Ma, T., Song, X., & Hwu, W. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599843 · Cited by 76
“For scientific text, SPECTER [12], an embedding model created using SciBERT [4], a variant of BERT, can be used.”
MethodMultiVerS: Improving scientific claim verification with weak supervision and full-document context
Wadden, D., Lo, K., Wang, L., Cohan, A., Beltagy, I., & Hajishirzi, H. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.6 · Cited by 60
“Domain adaptation for scientific text has been studied in a number of works, including Gururangan et al. (2020); Beltagy et al. (2019); Lee et al. (2020); Gu et al. (2021).”
BackgroundSelf-citationGuillarme, N. L., & Thuiller, W. (2021) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13778 · Cited by 64
SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction
Xiao, Y., Zhang, Z., Mao, Y., Yang, C., & Han, J. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.171 · Cited by 87
Chang, J. C., Zhang, A. X., Bragg, J., Head, A., Lo, K., Downey, D., & Weld, D. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580847 · Cited by 78
Self-citationWhen Geoscience Meets Foundation Models: Toward a general geoscience artificial intelligence system
Zhang, H., Xu, J., Cui, H., Li, L., Yang, Y., Tang, C., & Boers, N. (2024) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2024.3496478 · Cited by 82
“Several LLMs for geoscience have been developed based on BERT [97], SciBERT [186], and LLaMA [187].”
SciCap: Generating Captions for Scientific Figures
Hsu, T.-Y., Giles, C. L., & Huang, T.-H. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.277 · Cited by 59
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.12833 · Cited by 59
aeroBERT-Classifier: Classification of Aerospace Requirements Using BERT
Ray, A. T., Cole, B. F., Fischer, O. J. P., White, R. T., & Mavris, D. N. (2023) · Aerospace · doi:10.3390/aerospace10030279 · Cited by 71
BioREx: Improving biomedical relation extraction by leveraging heterogeneous datasets
Lai, P., Wei, C.-H., Luo, L., Chen, Q., & Lu, Z. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104487 · Cited by 70
Improving biomedical named entity recognition with syntactic information
Tian, Y., Wang, S., Song, Y., Xia, F., He, M., & Li, K. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03834-6 · Cited by 67
Covid-on-the-Web: Knowledge Graph and Services to Advance COVID-19 Research
Michel, F., Gandon, F., Ah-Kane, V., Bobasheva, A., Cabrio, E., Corby, O., Gazzotti, R., Giboin, A., Marro, S., Mayer, T., et al. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-62466-8_19 · Cited by 46
Searching for scientific evidence in a pandemic: An overview of TREC-COVID
Roberts, K. E., Alam, T., Bedrick, S. D., Demner‐Fushman, D., Lo, K., Soboroff, I. M., Voorhees, E. M., Wang, L. L., & Hersh, W. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103865 · Cited by 78
“This automatic system used SciBERT [46] to re-rank the output of a BM25 retrieval stage.”
Self-citationImproving accuracy of GPT-3/4 results on biomedical data using a retrieval-augmented language model
Soong, D., Sridhar, S., Si, H., Wagner, J.-S., Sá, A. C. C., Yu, C. Y., Karagoz, K., Guan, M., Kumar, S., Hamadeh, H., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000568 · Cited by 49
Train No Evil: Selective Masking for Task-Guided Pre-Training
Gu, Y., Zhang, Z., Wang, X., Liu, Z., & Sun, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.566 · Cited by 77
Jahan, I., Laskar, M. T. R., Peng, C., & Huang, J. (2023) · The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks · doi:10.18653/v1/2023.bionlp-1.30 · Cited by 40
Faithful AI in Medicine: A Systematic Review with Large Language Models and Beyond
Xie, Q., Schenck, E. J., Yang, H. S., Chen, Y., Peng, Y., & Wang, F. (2023) · doi:10.21203/rs.3.rs-3661764/v1 · Cited by 57
“The instruction prompt tuning used in Med-PaLM proves effective in improving its factuality, consistency, and safety.”
MethodCao, L., & Liu, Q. (2024) · ACM Computing Surveys · doi:10.1145/3686150 · Cited by 61
“Our keyword-based similarity analysis using the SciBERT pre-trained vector [Beltagy et al. , 2019] and domain-expert filtering of the 160k literature with some 12k on modeling comes up with the following brief statistics of extracted key-words about the COVID-19 business concerns, disciplinary and…”
MethodShort-Text Semantic Similarity (STSS): Techniques, Challenges and Future Perspectives
Amur, Z. H., Hooi, Y. K., Bhanbhro, H., Dahri, K., & Soomro, G. M. (2023) · Applied Sciences · doi:10.3390/app13063911 · Cited by 78
Jahan, I., Laskar, M. T. R., Peng, C., & Huang, J. (2023) · arXiv · doi:10.48550/arxiv.2306.04504 · Cited by 57
“The effective utilization of transformer-based (Vaswani et al., 2017) NLP models like BERT (Devlin et al., 2019) have also led to significant progress in the biomedical domain (Lee et al., 2020; Alsentzer et al., 2019; Beltagy et al., 2019; Gu et al., 2020; Peng et al., 2019) in recent years.”
MethodNeighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings
Ostendorff, M., Rethmeier, N., Augenstein, I., Gipp, B., & Rehm, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.802 · Cited by 56
Yang, M., Huang, L., Huang, H., Tang, H., Zhang, N., Yang, H., Wu, J., & Mu, F. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac326 · Cited by 67
Understanding Telecom Language Through Large Language Models
Bariah, L., Zou, H., Zhao, Q., Mouhouche, B., Bader, F., & Debbah, M. A. (2023) · Globecom · doi:10.1109/globecom54140.2023.10437725 · Cited by 74
“From a Domain Data [1] General Science Scientific publication [2] Multi-domain Patents Google Patents Public dataset [3] General domain Q&A Social commenting platforms [4] General domain Wikipedia articles [5] Cross-domain sentiment analysis Amazon reviews dataset [6] Customers delivery and pickup…”
Background
Metadata
- Authors
- Beltagy, I., Lo, K., & Cohan, A.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- ORCID
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 3613-3618
- OpenAlex
- W2970771982
- Semantic Scholar
- 156d217b0a911af97fa1b5a71dc909ccef7a8028
- Cited by
- 3,218
- References
- 33
- Citations by year
- 2026: 328 · 2025: 534 · 2024: 542 · 2023: 626 · 2022: 478 · 2021: 486 · 2020: 203 · 2019: 19
- FWCI
- 113.46
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- SciBERT, pre-trained language models, scientific texts, scientific publications, biomedical text, natural language processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.