SciBERT: A Pretrained Language Model for Scientific Text
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index; 9 of the 33 sources its claims cite were checked for support.
Abstract
Iz Beltagy, Kyle Lo, Arman Cohan. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
CoChat Summary
TL;DR
SciBERT is a BERT-based pretrained language model trained on a 3.17B-token corpus of 1.14M scientific papers, with an in-domain WordPiece vocabulary. It improves over BERT-Base across scientific NLP tasks and achieves new state-of-the-art results on several datasets.
Key findings
- SciBERT outperforms BERT-Base on scientific tasks by +2.11 F1 with finetuning and +2.43 F1 without finetuning.
- Domain breakdowns: biomedical +1.92 F1 with finetuning and +3.59 F1 without; computer science +3.55 F1 with finetuning and +1.13 F1 without; multidomain +0.49 F1 with finetuning and +0.93 F1 without.
- Finetuning beats frozen BERT embeddings by +3.25 F1 with SciBERT and +3.58 F1 with BERT-Base on average.
- Using SCIVOCAB instead of BASEVOCAB gives +0.60 F1 averaged across datasets: +0.76 F1 biomedical, +0.61 F1 computer science, +0.11 F1 multidomain. SciBERT achieves new SOTA on BC5CDR, ChemProt, EBM-NLP, ACL-ARC, the NER part of SciERC, and SciCite.
Methods
- SciBERT follows the BERT-Base architecture and is pretrained on a random sample of 1.14M Semantic Scholar papers: 18% computer science and 82% broad biomedical, using full text, averaging 154 sentences (2,769 tokens) per paper, for 3.17B tokens.
- The authors build SCIVOCAB, a 30K WordPiece vocabulary with 42% token overlap with BASEVOCAB, and train four variants: cased/uncased and BASEVOCAB/SCIVOCAB.
- Evaluation covers NER, PICO extraction, text classification, relation classification, and dependency parsing, using datasets including EBM-NLP, SciERC, ACL-ARC, SciCite, and Paper Field; results are macro F1 for most tasks, micro F1 for ChemProt, and LAS/UAS for DEP, averaged over multiple random seeds.
Limitations
- The paper states future work will release a BERT-Large analogue and experiment with different proportions of papers from each domain, noting these models are costly to train and aiming for a single resource useful across multiple domains.
- The paper notes SciBERT performs slightly worse than SOTA on JNLPBA, NCBI-disease, and GENIA, attributing these to comparison models: a BiLSTM-CRF ensemble trained on multiple NER datasets, BIOBERT trained on 18B biomedical tokens, and a parser using POS features not used here.
- Evident from the text: the pretraining corpus is limited to computer science and biomedical papers, so coverage of other scientific domains is not established; frozen-embedding experiments did not use extensive hyperparameter search; and SciERC relation results are not comparable to prior work because this paper uses gold entities while prior work performs joint entity and relation extraction.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
9 Valid · 0 Not valid · 16 Not checked · 29 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Yet while both BERT and ELMo have released pretrained models, they are still trained on general domain corpora such as news articles and Wikipedia.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites Devlin et al., 2019 matched to “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” (2019) source
- Partly supportedcites Peters et al., 2018 matched to “Deep Contextualized Word Representations” (2018) sourceExperimental — accuracy not yet measured
Section 3.4 says the ELMo biLM was pretrained on the 1B Word Benchmark, and the Introduction says the trained models were made publicly available. The paper does not discuss BERT or Wikipedia, or identify the benchmark text as news articles, so it does not support those parts of the sentence. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“As shown through ELMo (Peters et al., 2018), GPT (Radford et al., 2018) and BERT (Devlin et al., 2019), unsupervised pretraining of language models on large corpora significantly improves performance on many NLP tasks.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 1 Introduction
What the sources say
- Supportedcites Peters et al., 2018 matched to “Deep Contextualized Word Representations” (2018) source
Section 2 describes pretraining ELMo’s bidirectional language model on unlabeled data, and Section 3.4 identifies the large 1B Word Benchmark training corpus. Section 4 reports that adding ELMo improved results across six NLP benchmark tasks, with relative error reductions of 6–20%. This supports the claim as it relates to ELMo; the other cited models are not attributed to this source. (LLM, conf=0.92)
- Not matchedcites Radford et al., 2018
An author–year citation the paper’s reference list doesn’t settle. We don’t search these by name.
- Not checked — no full text of the sourcecites Devlin et al., 2019 matched to “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” (2019) source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“SCIBERT significantly outperformed BERT-Base and achieves new SOTA results on several of these tasks, even compared to some reported BIOBERT (Lee et al., 2019) results on biomedical tasks.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 7 Conclusion and Future Work
What the sources say
- Supportedcites Lee et al., 2019 matched to “BioBERT: a pre-trained biomedical language representation model for biomedical text mining” (2019) source
The citation identifies the source of the reported BioBERT results used as a comparator, rather than attributing the SciBERT comparison to Lee et al. The Experimental results (§4.3) report BioBERT performance on biomedical named-entity recognition, relation extraction, and question-answering tasks. (LLM, conf=0.90)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In Table 2, we compare SCIBERT results with reported BIOBERT results on the subset of datasets included in (Lee et al., 2019) BC5CDR and ChemProt, and performs similarly on JNLPBA despite being trained on a substantially smaller biomedical corpus.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 4.1 Biomedical Domain
What the sources say
- Supportedcites Lee et al., 2019 matched to “BioBERT: a pre-trained biomedical language representation model for biomedical text mining” (2019) source
The results tables in our copy report BioBERT scores for BC5CDR, ChemProt, and JNLPBA (section 6; page unavailable). The citation identifies the source of those reported BioBERT results; the comparison with SciBERT is the citing authors’ own conclusion. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In addition, SCIBERT outperforms the SOTA on SciCite (Co-han et al., 2019).”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 4.3 Multiple Domains
What the sources say
- Partly supportedcites Cohan et al., 2019 matched to “Structural Scaffolds for Citation Intent Classification in Scientific Publications” (2019) sourceExperimental — accuracy not yet measured
Table 4 (section 4.3 Results; page unavailable) shows that the paper’s BiLSTM-Attn model with ELMo and both scaffolds outperformed the listed previous SOTA on SciCite (84.0 versus 79.6 macro F1). It does not report a SciBERT result, so it does not support attributing that improvement to SciBERT. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
17 more claims
Create a free account to see all 54 claims and to verify the 13 references still open.
- Probably not a substantial claim. 17
- It may state two separate claims as one. 17
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“With their word-piece vocabulary, contextual embeddings, and self-supervised pre-training on large unlabelled corpora these models have essentially removed the need for most pre-processing beyond automatically-applied lowercasing. 14,31”
From Data extraction methods for systematic review (semi)automation: Update of a living systematic review (2025) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“SciBERT, introduced by Beltagy et al. [16], was the first language model tailored for scientific texts.”
From Evaluating Large Language Models for Structured Science Summarization in the Open Research Knowledge Graph (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Hence, we base our experiments on well-established large language models (LLM) from the general and scientific domain -RoBERTa (Liu et al., 2019), BioBERT (Lee et al., 2020) and SciBERT ( Beltagy et al., 2019) -with the recommended default hyperparameter settings.”
From NLPeer: A Unified Resource for the Computational Study of Peer Review (2023) · cites this paper as Beltagy et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Although the first experiments with BERT ( 39 ) were disappointing, fine-tuning SciBERT ( 40 ) led to the best results and is the current production architecture of the platform.”
From scite: a smart citation index that displays the context of citations and classifies their intent using deep learning (2021) · cites this paper as [40]
The cited source says what the claim attributes to it.
In the citing paper
“Similar observations have been made in prior work, e.g. Beltagy et al. (2019) found that pre-training BERT on text from research papers improved its performance on scientific tasks.”
From Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019) · cites this paper as Beltagy et al. (2019)
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 7×
“3617 Casing We follow Devlin et al. (2019) in using the cased models for NER and the uncased models for all other tasks.”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 5×
“Recent work on domain adaptation of BERT includes B IO B ERT (Lee et al., 2019) and C LINI - CAL B ERT (Alsentzer et al., 2019; Huang et al., 2019).”
MethodResultInfluentialGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · cited 2×
“We finetune for 2 to 5 epochs using a batch size of 32 and a learning rate of 5e-6, 1e5, 2e-5, or 5e-5 with a slanted triangular schedule (Howard and Ruder, 2018) which is equivalent to the linear warmup followed by linear decay (Devlin et al.”
Method
Cited by
Items 1,601–1,700 of 3,596
Syntactic-Aware Text Classification Method Embedding the Weight Vectors of Feature Words
Wang, M., Kim, J., & Yan, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3545877 · Cited by 7
Subset selection for domain adaptive pre-training of language model
Hwang, J., Lee, S.-D., Kim, H. Y., & Jeong, Y.-S. (2025) · Scientific Reports · doi:10.1038/s41598-025-94085-z · Cited by 9
“Therefore, we took mean absolute error (MAE) as a metric for the OCEAN factor regression, while we report the classification results using macro-F1 and micro-F1 by following Gururangan et al. 14 and Beltagy et al. 6 .”
Iana, A., Schmidt, F. D., Glavaš, G., & Paulheim, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88711-6_8 · Cited by 7
CDˆ2CR: Co-reference resolution across documents and domains
Ravenscroft, J., Clare, A. J., Cattan, A., Dagan, I., & Liakata, M. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.21 · Cited by 16
Document Retrieval for Precision Medicine Using a Deep Learning Ensemble Method
Liu, Z., Feng, J., Yang, Z., & Wang, L. (2021) · JMIR Medical Informatics · doi:10.2196/28272 · Cited by 7
“Team CCNL trained a SciBERT (Scientific Bidirectional Encoder Representations from Transformers) [19] to classify all documents.”
MethodInfluentialRavfogel, S., Taub-Tabib, H., & Goldberg, Y. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.25 · Cited by 7
“Neural models, and in particular large pre-trained language models (Devlin et al., 2019; Beltagy et al., 2019), excel at this kind of fuzzier, less-rigid similarity matching.”
Multimodal Graph-based Transformer Framework for Biomedical Relation Extraction
Pingali, S., Yadav, S., Dutta, P., & Saha, S. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.328 · Cited by 7
Task-adaptive Pre-training of Language Models with Word Embedding Regularization
Nishida, K., Nishida, K., & Yoshida, S. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.398 · Cited by 11
“, 2020), improves the performance on the task (Alsentzer et al., 2019; Beltagy et al., 2019; Chalkidis et al., 2020).”
BackgroundTell Me How to Survey: Literature Review Made Simple with Automatic Reading Path Generation
Ding, J., Xiang, T., Ou, Z., Zuo, W., Zhao, R., Lin, C., Zheng, Y., & Liu, B. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00322 · Cited by 10
Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration
Mu, E., Jabbour, S., Dalca, A. V., Guttag, J. V., Wiens, J., & Sjoding, M. W. (2022) · PLoS ONE · doi:10.1371/journal.pone.0263922 · Cited by 8
“For example, text data may need to be first processed through an existing language model, prior to additional training on the joint predictions of the underlying models [23].”
MethodZhang, Y., & Ma, Q. (2022) · Computational Linguistics · doi:10.1162/coli_a_00438 · Cited by 7
A structured method to achieve cognitive depth for medical device use error topic modeling
Levine, C. S., Knisely, B. M., Johnson, D., & Vaughn-Cooke, M. (2022) · Human Factors in Healthcare · doi:10.1016/j.hfh.2022.100016 · Cited by 7
A multi-view method of scientific paper classification via heterogeneous graph embeddings
Lv, Y., Xie, Z. Y., Zuo, X., & Song, Y. (2022) · Scientometrics · doi:10.1007/s11192-022-04419-1 · Cited by 8
“Then a paper vector is obtained by averaging all vectors of the paper’s words or by the vector of the paper’s first word (Beltagy et al., 2019; Zhang et al., 2020).”
BackgroundAn automatic hypothesis generation for plausible linkage between xanthium and diabetes
Syafiandini, A. F., Song, G., Ahn, Y., Kim, H., & Song, M. (2022) · Scientific Reports · doi:10.1038/s41598-022-20752-0 · Cited by 7
Chanaa, H., Nfaoui, E. H., & Loqman, C. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-29857-8_95 · Cited by 7
Comparing different search methods for the open access journal recommendation tool B!SON
Entrup, E., Eppelin, A., Ewerth, R., Hartwig, J., Tullney, M., Wohlgemuth, M., & Hoppe, A. (2023) · International Journal on Digital Libraries · doi:10.1007/s00799-023-00372-3 · Cited by 7
“Thiseliminatesoptionssuchascommonlyused transformer-based models for scientific language (e.g., SciB-ERT [44]).”
ConfliBERT-Arabic: A Pre-trained Arabic Language Model for Politics, Conflicts and Violence
Alsarra, S., Abdeljaber, L., Yang, W., Zawad, N., Khan, L. R., Brandt, P. T., Osorio, J., & D'Orazio, V. J. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_011 · Cited by 8
Joint extraction of biomedical overlapping triples through feature partition encoding
Zhu, Q., Cheng, H., Meng, Y., Yang, H., & Zhao, W. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122723 · Cited by 7
Scientific Text Mining and Knowledge Graphs
Jiang, M., & Shang, J. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3406465 · Cited by 9
A Smart System to Generate and Validate Question Answer Pairs for COVID-19 Literature
Bhambhoria, R. V., Feng, L. Y., Sepehr, D., Chen, J., Cowling, C., Koçak, S. A., & Dolatabadi, E. (2020) · Workshop on Scholarly Document Processing (SDP) · doi:10.18653/v1/2020.sdp-1.4 · Cited by 7
End-to-End Fine-Grained Neural Entity Recognition of Patients, Interventions, Outcomes
Dhrangadhariya, A., Aguilar, G., Solorio, T., Hilfiker, R., & Müller, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85251-1_6 · Cited by 10
Concept recognition as a machine translation problem
Boguslav, M. R., Hailu, N., Bada, M. A., Baumgartner, W. A., & Hunter, L. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04141-4 · Cited by 8
“BERT [42], the science-specific language model SciBERT [43], and the biomedicine-specific language model BioBERT [15], all require little fine-tuning to perform well for named entity recognition.”
MethodRavanmehr, V., Blau, H., Cappelletti, L., Fontana, T., Carmody, L., Coleman, B., George, J., Reese, J., Joachimiak, M. P., Bocci, G., et al. (2021) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqab113 · Cited by 7
“The word2vec step could be replaced by more advanced word embedding methods such as Bidirectional Encoder Representations from Transformers (BERT) (52), including the SciBERT version trained on the scientific literature (53).”
MethodDiscovering Interdisciplinary Research Based on Neural Networks
He, T., Fu, W., Xu, J., Zhang, Z., Zhou, J., Yin, Y., & Xie, Z. (2022) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2022.908733 · Cited by 7
“We found that the majority of errors produced by the SciBERT-based method and the IRD-BERT-based method fell into the fi rst category, but the majority of errors produced by the Word2Vec-based method fell into the second category.”
BackgroundMethodInfluentialClinical Application of Detecting COVID-19 Risks: A Natural Language Processing Approach
Bashir, S. R., Raza, S., Kocaman, V., & Qamar, U. (2022) · Viruses · doi:10.3390/v14122761 · Cited by 12
“The BioBERT is quite generalizable compared to other BERT-based methods, the SciBERT is initially trained on scientific data (not clinical) [29], and, CT-BERT is pre-trained on social media data, so they perform differently with different entity types.”
MethodInfluentialCompositional Zero-Shot Domain Transfer with Text-to-Text Models
Liu, F., Liu, Q., Bannur, S., Pérez‐García, F., Usuyama, N., Zhang, S., Naumann, T., Nori, A., Poon, H., Alvarez-Valle, J., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00585 · Cited by 11
“Similar to PubMedBERT, SciBERT (Beltagy et al., 2019) pretrains from scratch on a mix of both PMC and computer science publications.”
MethodWeakly Supervised Named Entity Tagging with Learnable Logical RulesPreprint
Li, J., Ding, H., Shang, J., McAuley, J., & Feng, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.02282 · Cited by 10
“In our experiments, we use SciBert for two biomedical datasets and Bert for CoNLL2003 dataset.”
BackgroundMethodOverview and Insights from the SciVer Shared Task on Scientific Claim VerificationPreprint
Wadden, D., & Lo, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.08188 · Cited by 7
“In fact, Wadden et al. (2020) even reports higher performance by RoBERTa- large than SciBERT, where the former has 3 times more parameters, despite the latter being trained on in-domain data.”
MethodSelf-citationExtracting Semantics from Maintenance RecordsPreprint
Dixit, S., Mulwad, V., & Saxena, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.05454 · Cited by 7
“Named Entity Recognition systems have also been developed for specific scientific domains such as medical (Xu et al., 2017), biomedical (Lee et al. 2019) and computer science (Beltagy, Lo, and Cohan, 2019) to name a few.”
Zhang, T., Cai, Z., Wang, C., Qiu, M., Yang, B., & He, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.08983 · Cited by 12
“(1) BioBERT ( Lee et al., 2020), BlueBERT (Peng et al., 2019), SCIBERT ( Beltagy et al., 2019) and ClinicalBert (Huang et al., 2019) apply continual learning on (2) PubMedBERT (Gu et al., 2020) learns weights from scratch using PubMed data to obtain an in-domain vocabulary, alleviating the out-of-vocabulary (OOV) prob…”
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERTPreprint
Meng, Z., Liu, F., Clark, T. H., Shareghi, E., & Collier, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.04810 · Cited by 9
“, 2020), SciBERT (Beltagy et al., 2019) and PubMedBERT (Gu et al.”
BackgroundMethodInfluentialAptaBERT: Predicting aptamer binding interactionsPreprint
Morsch, F., Umasankar, I. L., Moreta, L. S., Latawa, P., Lange, D. B., Wengel, J., Konjen, H., & Code, C. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.24.568626 · Cited by 11
“There are also domain-specific adaptations like ClinicalBERT [66] for medical texts, SciBERT [67] for scientific literature, LegalBERT [68] for legal documents, FinBERT [69] for financial discourse, and BioBERT [70] for biomedical text.”
BackgroundWithdrawn: Large Language Models in Healthcare Decision Support: A ReviewPreprint
Vavekanand, R., Karttunen, P., Xu, Y., Milani, S., & Li, H. (2024) · Preprints.org · doi:10.20944/preprints202407.1842.v1 · Cited by 14
Temporal Graph Neural Network-Powered Paper Recommendation on Dynamic Citation NetworksPreprint
Shen, J., Haqqani, M. A. A., Hu, B., Huang, C., Xie, X., Lee, T. J., & Zhang, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.15371 · Cited by 12
“Memory Initialization: We compared memory initialization using semantic information from paper abstracts and titles (via SciBERT[22]) against a structure-only approach.”
MethodUnmasking the Reality of PII Masking Models: Performance Gaps and the Call for AccountabilityPreprint
Singh, D., & Narayanan, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.12308 · Cited by 7
Abdaoui, H., Barki, C., Dergaa, I., Tlili, K., Ceylan, H. İ., Bragazzi, N. L., de Giorgio, A., Ben Salah, R., & Rahmouni, H. B. (2025) · Bioengineering · doi:10.3390/bioengineering13010030 · Cited by 6
“Our precision of 0.969 and recall of 0.958 exceed the ranges of 0.84 to 0.87 reported for the SciBERT and BlueBERT variants [40], indicating that BioBERT pre-training on PubMed abstracts provides particularly effective initialization for medical entity extraction tasks.”
InfluentialMitigating Sycophancy in Large Language Models via Direct Preference Optimization
Khan, A. A., Alam, S., Wang, X., Khan, A. F., Neog, D. R., & Anwar, A. (2024) · 2024 IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825538 · Cited by 6
LitFM: A Retrieval Augmented Structure-aware Foundation Model For Citation Graphs
Zhang, J., Maatouk, A., Chen, J., Bui, N., Xie, Q., Tassiulas, L., Xu, H., Shao, J., & Ying, R. (2025) · Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · doi:10.1145/3711896.3737028 · Cited by 6
“Efforts such as SciBERT [4], BioBERT [32], PubMed-BERT [16], SciMult [66] aim for general understanding of literature but are only applicable for discriminative tasks due to their emphasis on representation learning.”
InfluentialBharti, P. K., Agarwal, M., & Ekbal, A. (2024) · Scientometrics · doi:10.1007/s11192-024-04938-z · Cited by 6
“SciBERT displays 100% accuracy for the Highly constructive label; however, the F1-score (90.0”
InfluentialIdentifying potentially disruptive research via a comparative power-based large model
Huang, S., Lu, W. P., Xu, Z., Cheng, Q., Yang, J., & Huang, Y. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104207 · Cited by 7
A multifaceted approach to drug–drug interaction extraction with fusion strategies
Tran, B.-H., Hoang, H.-M., Nguyen, B.-N., Can, D.-C., & Le, H.-Q. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104874 · Cited by 7
Mf-cite: citation intent classification in scientific papers based on multi-feature fusion
Xu, X., Xie, Y., Zhao, X., & Liu, Y. (2025) · Scientometrics · doi:10.1007/s11192-025-05365-4 · Cited by 7
“Zhang et al. (2022) used native information related to citation contexts, such as section name and paper title, and designed three different network structures based on LSTM (Alikaniotis et al., 2016), SciBERT (Belt-agy et al., 2019), and T5 (Raffel et al., 2020) models respectively to improve the performance of the m…”
InfluentialLLMs in Citation Intent Classification: Progress, Precision, and Reproducibility Challenges
Fogelson, A., Trišović, A., & Thompson, N. (2025) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3736731.3746137 · Cited by 9
“Pretrained language models (PLMs) such as SciBERT have improved classification accuracy through task-specific fine-tuning [2], yet challenges persist due to data scarcity and the inherent ambiguity of citation labels.”
Danish, A. A. K., & Chatterjee, S. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130068 · Cited by 8
Han, B., Mathrani, A., Wanniarachchi, V. U., & Sušnjak, T. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10110 · Cited by 9
Mortezaagha, P., Shaw, J., Sun, B., & Rahgozar, A. (2026) · BMC Medical Research Methodology · doi:10.1186/s12874-026-02847-8 · Cited by 8
“SciB-ERT, trained on large scientific corpora, has become a widely used backbone for entity extraction and classification in scholarly text (Beltagy et al, 2019).”
Casteleiro, M. A., Maroto, N., Wroe, C., Torrado, C. S., Henson, C., Diz, J. D., Fernandez-Prieto, M. J., Furmston, T., Fernandez, D. M., Kulshrestha, M., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-91100-3_14 · Cited by 7
Chen, Q., Yao, H. L., Zhou, D., Li, S., & Dong, L. (2023) · Computers & Geosciences · doi:10.1016/j.cageo.2023.105416 · Cited by 6
Trust Issues: Discrepancies in Trustworthy AI Keywords Use in Policy and Research
Toney, A., Curlee, K. J., & Probasco, E. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659035 · Cited by 8
“The AI classifier uses arXiv publications as training data, as arXiv publications receive author assigned topic labels, and the SciBert language model trained on Semantic Scholar [6].”
MethodIncorporating Part of Speech Information in span representation for Named Entity Recognition
Cui, Z., Guo, Z., Lan, Y., Xiao-mei, L., & Dai, H. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111844 · Cited by 7
Leveraging multiple control codes for aspect-controllable related paper recommendation
Long, K., Li, S., Tang, J., & Wang, T. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103879 · Cited by 7
He, S., Du, W., Peng, X., & Li, X. (2024) · Chemical Engineering Science · doi:10.1016/j.ces.2024.120916 · Cited by 10
Xiong, D., Li, X., Gao, L., & Gao, Y. (2024) · IET Collaborative Intelligent Manufacturing · doi:10.1049/cim2.70003 · Cited by 6
Chen, G., Chen, P., Wang, Q., Li, H., Zhou, X., Xibin, W., Yu, A., & Deng, X. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112777 · Cited by 6
DOME: Directional medical embedding vectors from Electronic Health Records
Wen, J., Xue, H., Rush, E. N., Panickan, V. A., Cai, T. A., Zhou, D., Ho, Y.-L. A., Costa, L., Begoli, E., Hong, C., et al. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104768 · Cited by 6
Emerging horizons of AI in pharmaceutical research
Bachhar, S., Kumar, S. P. J., Dutta, B., & Das, S. (2025) · Advances in pharmacology · doi:10.1016/bs.apha.2025.01.016 · Cited by 6
Advancing Scientific Text Classification: Fine-Tuned Models with Dataset Expansion and Hard-Voting
Rostam, Z. R. K., & Kertész, G. (2025) · IEEE World Symposium on Applied Machine Intelligence and Informatics (SAMI) · doi:10.1109/sami63904.2025.10883153 · Cited by 6
“[18]–[24].”
Wang, Z., Wang, N., Zhang, H., Wang, Z., Wang, Z., Ding, J., & Chen, H. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104121 · Cited by 6
ForestryBERT: A pre-trained language model with continual learning adapted to changing forestry text
Tan, J., Zhang, H., Yang, J., Liu, Y., Zheng, D., & Liu, X. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113706 · Cited by 8
Algorithms in the orchard: An embedding-based expert answering system for apple rust
Anand, A. V., Shen, J. G., Cremers, A. B., & Jacobs, M. D. (2025) · Smart Agricultural Technology · doi:10.1016/j.atech.2025.101069 · Cited by 6
RMAF: A replay method based on active forgetting for continual learning
Qiu, H., Feng, J., Zhao, L., Gu, C., Yu, H., Zhang, Y., & Wang, Z. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131098 · Cited by 8
A Survey of Long-Document Retrieval in the PLM and LLM Era
Li, M., Zhang, Y., Lv, T., Zhao, S., Luo, M., Nie, E., & Zhou, G. (2026) · ACM Transactions on Information Systems · doi:10.1145/3849382 · Cited by 8
Overview and Insights from Scope Detection of the Peer Review Articles Shared Tasks 2021
Reddy, S. M., & Saini, N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75015-2_7 · Cited by 6
Position-aware Joint Entity and Relation Extraction with Attention Mechanism
Zhang, C., Gao, S., Wang, H., & Zhang, W. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/624 · Cited by 9
A Lightweight CNN and Class Weight Balancing on Chest X-ray Images for COVID-19 Detection
Alduaiji, N., Algarni, A. D., Hamza, S. H. A., Abdel-Azim, G., & Hamam, H. (2022) · Electronics · doi:10.3390/electronics11234008 · Cited by 6
Zhao, Y., & Tsubota, T. (2023) · JMIR Medical Informatics · doi:10.2196/39876 · Cited by 8
“Review articles were automatically identified by a section classification model [19] trained on the PubMed 200k data set [20], which classified sentences in the abstracts into 5 sections (introduction, objective, method, result, and conclusion).”
Examine the Effectiveness of Patent Embedding-Based Company Comparison Method
Ha, T., & Lee, J.-M. (2023) · IEEE Access · doi:10.1109/access.2023.3251664 · Cited by 6
Keyword Embeddings for Query Suggestion
Gabín, J., Ares, M. E., & Parapar, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28244-7_22 · Cited by 7
Thushari, P. D., Niazi, S., & Meena, S. (2023) · IEEE International Conference for Convergence in Technology (I2CT) · doi:10.1109/i2ct57861.2023.10126262 · Cited by 7
“We have used ClinicalBERT, Sci-BERT, RoBERTa, and BERT-based models to compare the performances of the generic models (RoBERTa and BERT) versus domain-specific ones.”
MethodInfluentialChild-Sum EATree-LSTMs: enhanced attentive Child-Sum Tree-LSTMs for biomedical event extraction
Wang, L., Cao, H., Yuan, L., Guo, X., & Cui, Y. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05336-7 · Cited by 6
“The network architecture employs the pre-training model SCIBERT [26] to initialize the word embedding.”
MethodGraph-Enhanced Biomedical Abstractive Summarization Via Factual Evidence Extraction
Frisoni, G., Italiani, P., Moro, G., Bartolini, I., Boschetti, M. A., & Carbonaro, A. (2023) · SN Computer Science · doi:10.1007/s42979-023-01867-1 · Cited by 7
“Starting from SciBERT contextual representations [53], DEM enumerates all the possible text spans in a sentence up to a certain window length, then executes a joint detection and classification flow of (1) entities and triggers, (2) roles, (3) events and modifiers, through custom layers.”
BackgroundMethodGlobal information-aware argument mining based on a top-down multi-turn QA model
Liu, B., Schlegel, V., Thompson, P., Batista-Navarro, R., & Ananiadou, S. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103445 · Cited by 12
Ramamonjison, R., Yu, T. T., Xing, L., Mostajabdaveh, M., Li, X., Fu, X., Han, X., Chen, Y., Li, R., Mao, K., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.45 · Cited by 8
Multilingual Sequence-to-Sequence Models for Hebrew NLP
Eyal, M., Noga, H., Aharoni, R., Szpektor, I., & Tsarfaty, R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.487 · Cited by 6
Alharbi, R., & Al-Muhtasab, H. (2022) · Arabic Natural Language Processing Workshop (WANLP) · doi:10.18653/v1/2022.wanlp-1.30 · Cited by 6
New Siamese Neural Networks for Text Classification and Ontologies Alignment
Menad, S., Laddada, W., Abdeddaïm, S., & Soualmia, L. F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44355-8_2 · Cited by 7
Yamazaki, T., & Sakata, I. (2023) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2023.3327209 · Cited by 6
Agriculture Named Entity Recognition—Towards FAIR, Reusable Scholarly Contributions in Agriculture
D’Souza, J. (2024) · Knowledge · doi:10.3390/knowledge4010001 · Cited by 11
“…original BERT model trained on the BookCorpus [76] plus English Wikipedia; and a pretrained variant released over scientific text called SciBERT [77] trained on 1.14 M papers from Semantic Scholar [4] which consists of 18% papers from the computer science domain and 82% from the biomedical…”
MethodInfluentialCHEMFUZZ: Large Language Models-Assisted Fuzzing for Quantum Chemistry Software Bug Detection
Qiu, F., Ji, P., Hua, B., & Wang, Y. (2023) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c60940.2023.00104 · Cited by 11
“Beltagy et al. [10] release a pre-trained BERT [12] language model called SciBERT, which brings up a better performance on downstream scientific Natural Language Processing tasks.”
MetaTron: advancing biomedical annotation empowering relation annotation and collaboration
Irrera, O., Marchesin, S., & Silvello, G. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05730-9 · Cited by 10
“All the ML models adopt SciBERT [84], a pre-trained language model based on BERT [85].”
MethodZhang, Y., Pan, L., Yang, Q., Li, T., & Chen, Z. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3384407 · Cited by 7
“We start from the CLIP-ViT-B model [7] as the vision encoder, the RoBERTa-base [38] as the language encoder, with the multi-modal fusion module provided by M 3 AE [6].”
What can rhetoric bring us? Incorporating rhetorical structure into neural related work generation
Wang, P., Li, S., Tang, J., & Wang, T. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123781 · Cited by 6
Hang, T., Feng, J., Wang, Y., & Yan, L. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05448-4 · Cited by 6
“For the DocRED dataset, we utilized the BERT-base [42] or Roberta-large models[43]; For the CDR and GDA datasets, we utilized the SciBERT-base model [44], which is pre-trained on the scientific publication corpora.”
Location-enhanced syntactic knowledge for biomedical relation extraction
Zhang, Y., Yang, Z., Yang, Y., Lin, H., & Wang, J. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104676 · Cited by 9
Scientific Software Citation Intent Classification Using Large Language Models
Istrate, A.-M., Fisher, J., Yang, X., Moraw, K., Li, K., Li, D., & Klein, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-65794-8_6 · Cited by 10
Steiert, D., Wittig, C., Banerjee, P., Preißner, R., & Szulcek, R. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012417 · Cited by 6
“Furthermore, the incorporation of advanced neural network models such as BERT and its pre-trained version SciBERT [43,44], which understand complex language contexts and specialized scientific vocabularies, as well as improvements to the original word2-vec word embedding model [22] that add…”
Hierarchical Tree-structured Knowledge Graph For Academic Insight Survey
Li, J., Huy, P., Gu, W., Ota, K., & Hasegawa, S. (2024) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista62901.2024.10683856 · Cited by 7
“Finally, sentences with the same ‘Issue Resolved’ and ‘Issue Finding’ labels in each article form the ‘insight sentence’ of that article.”
BackgroundPhysBERT: A text embedding model for physics scientific literature
Hellert, T., Montenegro, J., & Pollastro, A. (2024) · APL Machine Learning · doi:10.1063/5.0238090 · Cited by 13
Pathak, P., & Rana, P. S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78107-0_10 · Cited by 7
Knowledge-augmented pre-trained language models for biomedical relation extraction
Sänger, M., & Leser, U. (2025) · BMC Bioinformatics · doi:10.1186/s12859-025-06262-6 · Cited by 7
“Studies reporting performance improvements leverage either PubMedBERT [20], BioBERT [30] or SciBERT [8], In contrast, we also include larger PLMs in our evaluation.”
COVIDSeer: Extending the CORD-19 Dataset
Rohatgi, S., Karishma, Z., Chhay, J., Keesara, S. R. R., Wu, J. F., Caragea, C., & Giles, C. L. (2020) · ACM Symposium on Document Engineering (DocEng) · doi:10.1145/3395027.3419597 · Cited by 7
Yu, J., Aduragba, O. T., Sun, Z., Black, S., Stewart, C. D., Shi, L., & Cristea, A. I. (2020) · International Conference on Computer Science & Education (ICCSE) · doi:10.1109/iccse49874.2020.9201631 · Cited by 6
Ionescu, M., Avram, A.-M., Dima, G.-A., Cercel, D.-C., & Dascălu, M. (2020) · Cited by 6
OntoSem: an Ontology Semantic Representation Methodology for Biomedical Domain
Zhao, L., Wang, J., Cheng, L., & Wang, C. (2020) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm49941.2020.9313128 · Cited by 7
“Applying the transfer learning technique, various BERT versions for biomedical domains such as bioBERT[7], sci-BERT[8], and NCBI-BERT[9] are trained on textual database such as PubMed abstract(https://www.ncbi.nlm.nih.gov/pubmed/), offering state-of-the-art results in various biomedical tasks.”
MethodKnowGraph@IITK at SemEval-2021 Task 11: Building Knowledge Graph for NLP Research
Shailabh, S., Chaurasia, S., & Modi, A. (2021) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2021.semeval-1.57 · Cited by 6
Arora, H., Ghosal, T., Kumar, S., Patwal, S. S., & Gooch, P. (2021) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2021.semeval-1.61 · Cited by 6
A Survey of Citation Recommendation Tasks and Methods
Medić, Z., & Šnajder, J. (2021) · Journal of Computing and Information Technology · doi:10.20532/cit.2020.1005160 · Cited by 12
Uncertainty-based Self-training for Biomedical Keyphrase Extraction
Gero, Z., & Ho, J. C. (2021) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi50953.2021.9508592 · Cited by 7
Separating Retention from Extraction in the Evaluation of End-to-end Relation Extraction
Taillé, B., Guigue, V., Scoutheeten, G., & Gallinari, P. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.816 · Cited by 9
“They rely on various pretrained LMs and for a fairer comparison, we use BERT (Devlin et al., 2019) on ACE05 and CoNLL04 and SciBERT (Beltagy et al., 2019) on SciERC 1 .”
BackgroundMethodMenin, A., Michel, F., Gandon, F. L., Gazzotti, R., Cabrio, E., Corby, O., Giboin, A., Marro, S., Mayer, T., Villata, S., et al. (2021) · Quantitative Science Studies · doi:10.1162/qss_a_00164 · Cited by 6
Metadata
- Authors
- Beltagy, I., Lo, K., & Cohan, A.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- ORCID
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 3613-3618
- OpenAlex
- W2970771982
- Semantic Scholar
- 156d217b0a911af97fa1b5a71dc909ccef7a8028
- Cited by
- 3,218
- References
- 33
- Citations by year
- 2026: 328 · 2025: 534 · 2024: 542 · 2023: 626 · 2022: 478 · 2021: 486 · 2020: 203 · 2019: 19
- FWCI
- 113.46
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- SciBERT, pre-trained language models, scientific texts, scientific publications, biomedical text, natural language processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.