SciBERT: A Pretrained Language Model for Scientific Text
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index; 9 of the 33 sources its claims cite were checked for support.
Abstract
Iz Beltagy, Kyle Lo, Arman Cohan. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
CoChat Summary
TL;DR
SciBERT is a BERT-based pretrained language model trained on a 3.17B-token corpus of 1.14M scientific papers, with an in-domain WordPiece vocabulary. It improves over BERT-Base across scientific NLP tasks and achieves new state-of-the-art results on several datasets.
Key findings
- SciBERT outperforms BERT-Base on scientific tasks by +2.11 F1 with finetuning and +2.43 F1 without finetuning.
- Domain breakdowns: biomedical +1.92 F1 with finetuning and +3.59 F1 without; computer science +3.55 F1 with finetuning and +1.13 F1 without; multidomain +0.49 F1 with finetuning and +0.93 F1 without.
- Finetuning beats frozen BERT embeddings by +3.25 F1 with SciBERT and +3.58 F1 with BERT-Base on average.
- Using SCIVOCAB instead of BASEVOCAB gives +0.60 F1 averaged across datasets: +0.76 F1 biomedical, +0.61 F1 computer science, +0.11 F1 multidomain. SciBERT achieves new SOTA on BC5CDR, ChemProt, EBM-NLP, ACL-ARC, the NER part of SciERC, and SciCite.
Methods
- SciBERT follows the BERT-Base architecture and is pretrained on a random sample of 1.14M Semantic Scholar papers: 18% computer science and 82% broad biomedical, using full text, averaging 154 sentences (2,769 tokens) per paper, for 3.17B tokens.
- The authors build SCIVOCAB, a 30K WordPiece vocabulary with 42% token overlap with BASEVOCAB, and train four variants: cased/uncased and BASEVOCAB/SCIVOCAB.
- Evaluation covers NER, PICO extraction, text classification, relation classification, and dependency parsing, using datasets including EBM-NLP, SciERC, ACL-ARC, SciCite, and Paper Field; results are macro F1 for most tasks, micro F1 for ChemProt, and LAS/UAS for DEP, averaged over multiple random seeds.
Limitations
- The paper states future work will release a BERT-Large analogue and experiment with different proportions of papers from each domain, noting these models are costly to train and aiming for a single resource useful across multiple domains.
- The paper notes SciBERT performs slightly worse than SOTA on JNLPBA, NCBI-disease, and GENIA, attributing these to comparison models: a BiLSTM-CRF ensemble trained on multiple NER datasets, BIOBERT trained on 18B biomedical tokens, and a parser using POS features not used here.
- Evident from the text: the pretraining corpus is limited to computer science and biomedical papers, so coverage of other scientific domains is not established; frozen-embedding experiments did not use extensive hyperparameter search; and SciERC relation results are not comparable to prior work because this paper uses gold entities while prior work performs joint entity and relation extraction.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
9 Valid · 0 Not valid · 16 Not checked · 29 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Yet while both BERT and ELMo have released pretrained models, they are still trained on general domain corpora such as news articles and Wikipedia.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites Devlin et al., 2019 matched to “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” (2019) source
- Partly supportedcites Peters et al., 2018 matched to “Deep Contextualized Word Representations” (2018) sourceExperimental — accuracy not yet measured
Section 3.4 says the ELMo biLM was pretrained on the 1B Word Benchmark, and the Introduction says the trained models were made publicly available. The paper does not discuss BERT or Wikipedia, or identify the benchmark text as news articles, so it does not support those parts of the sentence. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“As shown through ELMo (Peters et al., 2018), GPT (Radford et al., 2018) and BERT (Devlin et al., 2019), unsupervised pretraining of language models on large corpora significantly improves performance on many NLP tasks.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 1 Introduction
What the sources say
- Supportedcites Peters et al., 2018 matched to “Deep Contextualized Word Representations” (2018) source
Section 2 describes pretraining ELMo’s bidirectional language model on unlabeled data, and Section 3.4 identifies the large 1B Word Benchmark training corpus. Section 4 reports that adding ELMo improved results across six NLP benchmark tasks, with relative error reductions of 6–20%. This supports the claim as it relates to ELMo; the other cited models are not attributed to this source. (LLM, conf=0.92)
- Not matchedcites Radford et al., 2018
An author–year citation the paper’s reference list doesn’t settle. We don’t search these by name.
- Not checked — no full text of the sourcecites Devlin et al., 2019 matched to “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” (2019) source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“SCIBERT significantly outperformed BERT-Base and achieves new SOTA results on several of these tasks, even compared to some reported BIOBERT (Lee et al., 2019) results on biomedical tasks.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 7 Conclusion and Future Work
What the sources say
- Supportedcites Lee et al., 2019 matched to “BioBERT: a pre-trained biomedical language representation model for biomedical text mining” (2019) source
The citation identifies the source of the reported BioBERT results used as a comparator, rather than attributing the SciBERT comparison to Lee et al. The Experimental results (§4.3) report BioBERT performance on biomedical named-entity recognition, relation extraction, and question-answering tasks. (LLM, conf=0.90)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In Table 2, we compare SCIBERT results with reported BIOBERT results on the subset of datasets included in (Lee et al., 2019) BC5CDR and ChemProt, and performs similarly on JNLPBA despite being trained on a substantially smaller biomedical corpus.”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 4.1 Biomedical Domain
What the sources say
- Supportedcites Lee et al., 2019 matched to “BioBERT: a pre-trained biomedical language representation model for biomedical text mining” (2019) source
The results tables in our copy report BioBERT scores for BC5CDR, ChemProt, and JNLPBA (section 6; page unavailable). The citation identifies the source of those reported BioBERT results; the comparison with SciBERT is the citing authors’ own conclusion. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In addition, SCIBERT outperforms the SOTA on SciCite (Co-han et al., 2019).”
§ SCIBERT: A Pretrained Language Model for Scientific Text > 4.3 Multiple Domains
What the sources say
- Partly supportedcites Cohan et al., 2019 matched to “Structural Scaffolds for Citation Intent Classification in Scientific Publications” (2019) sourceExperimental — accuracy not yet measured
Table 4 (section 4.3 Results; page unavailable) shows that the paper’s BiLSTM-Attn model with ELMo and both scaffolds outperformed the listed previous SOTA on SciCite (84.0 versus 79.6 macro F1). It does not report a SciBERT result, so it does not support attributing that improvement to SciBERT. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
17 more claims
Create a free account to see all 54 claims and to verify the 13 references still open.
- Probably not a substantial claim. 17
- It may state two separate claims as one. 17
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“With their word-piece vocabulary, contextual embeddings, and self-supervised pre-training on large unlabelled corpora these models have essentially removed the need for most pre-processing beyond automatically-applied lowercasing. 14,31”
From Data extraction methods for systematic review (semi)automation: Update of a living systematic review (2025) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“SciBERT, introduced by Beltagy et al. [16], was the first language model tailored for scientific texts.”
From Evaluating Large Language Models for Structured Science Summarization in the Open Research Knowledge Graph (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Hence, we base our experiments on well-established large language models (LLM) from the general and scientific domain -RoBERTa (Liu et al., 2019), BioBERT (Lee et al., 2020) and SciBERT ( Beltagy et al., 2019) -with the recommended default hyperparameter settings.”
From NLPeer: A Unified Resource for the Computational Study of Peer Review (2023) · cites this paper as Beltagy et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Although the first experiments with BERT ( 39 ) were disappointing, fine-tuning SciBERT ( 40 ) led to the best results and is the current production architecture of the platform.”
From scite: a smart citation index that displays the context of citations and classifies their intent using deep learning (2021) · cites this paper as [40]
The cited source says what the claim attributes to it.
In the citing paper
“Similar observations have been made in prior work, e.g. Beltagy et al. (2019) found that pre-training BERT on text from research papers improved its performance on scientific tasks.”
From Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019) · cites this paper as Beltagy et al. (2019)
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 7×
“3617 Casing We follow Devlin et al. (2019) in using the cased models for NER and the uncased models for all other tasks.”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 5×
“Recent work on domain adaptation of BERT includes B IO B ERT (Lee et al., 2019) and C LINI - CAL B ERT (Alsentzer et al., 2019; Huang et al., 2019).”
MethodResultInfluentialGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · cited 2×
“We finetune for 2 to 5 epochs using a batch size of 32 and a learning rate of 5e-6, 1e5, 2e-5, or 5e-5 with a slanted triangular schedule (Howard and Ruder, 2018) which is equivalent to the linear warmup followed by linear decay (Devlin et al.”
Method
Cited by
Items 901–1,000 of 3,596
Knowledge Graph-Empowered Materials Discovery
Zhao, X., Greenberg, J., McClellan, S., Hu, Y., Lopez, S. A., Saikin, S. K., Hu, X. T., & An, Y. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671503 · Cited by 24
“More recently, attention-based pre-trained language models [32], [33] have achieved superior performance in various NLP tasks.”
BackgroundSarrouti, M., Tao, C., & Randriamihaja, Y. M. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.37 · Cited by 29
A multilingual BERT-based classification of reviews for enhanced visitors’ experience analysis
Ricciardi, R., & Manisera, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-09418-9 · Cited by 17
“Furthermore, several BERT-like architectures have been proposed, e.g. RoBERTa 10 , trained on larger corpora than BERT, DistilBERT 11 and ALBERT 12 , for applications with limited computational resources, and SciBERT, pre-trained on 1,4 million scientific papers from Semantic Scholar 13 .”
HOLMS: Alternative Summary Evaluation with Large Language Models
Mrabet, Y., & Demner‐Fushman, D. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.498 · Cited by 17
“In future works, we plan to test more recent language models such as T5 (Raffel et al., 2019) and SciBERT (Beltagy et al., 2019).”
MethodStocker, M., Heger, T., Schweidtmann, A. M., Ćwiek‐Kupczyńska, H., Penev, L., Dojchinovski, M., Willighagen, E., Vidal, M.-E., Turki, H., Balliet, D. P., et al. (2022) · Research Ideas and Outcomes · doi:10.3897/rio.8.e83789 · Cited by 17
Information retrieval and question answering: A case study on COVID-19 scientific literature
Otegi, A., Vicente, I. S., Saralegi, X., Peñas, A., Lozano, B., & Agirre, E. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.108072 · Cited by 25
“More specifically, we have used the SciBERT language representation model, which is a pre-trained language model based on BERT, but trained on a large corpus of scientific text, including text from the biomedical domain [52].”
Context-aware citation recommendation of scientific papers: comparative study, gaps and trends
Jebari, C., Herrera‐Viedma, E., & Cobo, M. J. (2023) · Scientometrics · doi:10.1007/s11192-023-04773-8 · Cited by 19
“Recently, Nicholson et al. (2021) developed a smart citation index called scite, 3 which is used to categorize citation contexts into three classes (supporting, contrasting, and mentioning) using the deep learning model SciBERT (Beltagy et al., 2019).”
MethodNechakhin, V., D’Souza, J., & Eger, S. (2024) · Information · doi:10.3390/info15060328 · Cited by 17
“This was followed by SPECTER, developed by Cohan et al. [11].”
MethodNandy, A., Sharma, S., Maddhashiya, S., Sachdeva, K., Goyal, P., & Ganguly, N. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.392 · Cited by 17
PolitePEER: does peer review hurt? A dataset to gauge politeness intensity in the peer reviews
Bharti, P. K., Navlakha, M., Agarwal, M., & Ekbal, A. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09662-3 · Cited by 37
“Nonetheless, SciBERT, being fine-tuned to focus on scientific jargon, might have literally interpreted the sentence and thus classified it as polite (4).”
InfluentialContextualised segment-wise citation function classification
Jiang, X., & Chen, J. (2023) · Scientometrics · doi:10.1007/s11192-023-04778-3 · Cited by 23
“Indeed, Beltagy et al. (2019) reported that their first SciBERT baseline performed very well on scicite , which contains only a single citance for each sample and thus only allows CFC at citance level.”
BackgroundMethodResultInfluentialRethinking Semi-supervised Learning with Language Models
Shi, Z., Tonolini, F., Αλέτρας, Ν., Yılmaz, E., Kazai, G., & Jiao, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.347 · Cited by 36
NLI4CT: Multi-Evidence Natural Language Inference for Clinical Trial Reports
Jullien, M., Valentino, M., Frost, H., O’Regan, P., Landers, D., & Freitas, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1041 · Cited by 54
Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding
Zhang, Y., Cheng, H., Shen, Z., Liu, X., Wang, Y., & Gao, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.820 · Cited by 28
Entity linking for English and other languages: a survey
Guellil, I., García‐Domínguez, A., Lewis, P. R., Hussain, S., & Smith, G. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-023-02059-2 · Cited by 21
Jointly Extracting Interventions, Outcomes, and Findings from RCT Reports with LLMs
Wadhwa, S., DeYoung, J., Nye, B., Amir, S., & Wallace, B. (2023) · PubMed · doi:10.48550/arxiv.2305.03642 · Cited by 33
“…in detecting PICO elements at the sentence level using LSTMs. Schmidt et al. (2020) proposed framing PICO extraction as a question-answering task and subsequently using transformer models, including SciBERT (Beltagy et al., 2019) — a masked language model pretrained on large-scale scientific data.”
MethodWhat’s New? Summarizing Contributions in Scientific Literature
Hayashi, H., Kryściński, W., McCann, B., Rajani, N. F., & Xiong, C. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.72 · Cited by 22
Task Compass: Scaling Multi-task Pre-training with Task Prefix
Zhang, Z., Wang, S., Yi‐chong, X., Fang, Y., Yu, W., Liu, Y., Zhao, H. Y., Zhu, C., & Zeng, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.416 · Cited by 25
Ad-hoc Document Retrieval using Weak-Supervision with BERT and GPT2
Mass, Y., & Roitman, H. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.343 · Cited by 21
Data Augmentation for Low-Resource Named Entity Recognition Using BacktranslationPreprint
Yaseen, U., & Langer, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.11703 · Cited by 17
“However, the average performance improvement due to data augmentation with SciBERT embeddings is lower as compared to the GloVe embeddings.”
MethodInfluentialFlexR: Few-shot Classification with Language Embeddings for Structured Reporting of Chest X-raysPreprint
Keicher, M., Zaripova, K., Czempiel, T., Mach, K., Khakzar, A., & Navab, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.15723 · Cited by 24
“Therefore, we replaced the image and text encoders with domain-specific pretrained encoders DenseNet121 (DN121) (Huang et al., 2017) pretrained on the detection of pathologies and SciBERT (SB) (Beltagy et al., 2019) pretrained on a large corpus of scientific text.”
MethodSercombe, J., Bryant, Z., & Wilson, J. (2025) · JMIR Formative Research · doi:10.2196/68666 · Cited by 16
“Recent developments in deep learning, particularly transformer-based models such as BERT and SciBERT, have improved the performance of such tasks by capturing more nuanced linguistic features [15-17].”
Steurer, B., Vanhaelen, Q., & Zhavoronkov, A. A. (2024) · The Journals of Gerontology Series A · doi:10.1093/gerona/glae006 · Cited by 18
“LLMs trained on scientific literature, biomedical, and clinical records have been released with BioMegatron (43), SciBERT (44), PubMedBERT (45), BioBERT (46), BioNLP (47), and GatorTron (48), to name a few (Table 1).”
MethodFine-tuning BERT Models for Keyphrase Extraction in Scientific Articles
Lim, Y., Seo, D., & Jung, Y. (2020) · JOURNAL OF ADVANCED INFORMATION TECHNOLOGY AND CONVERGENCE · doi:10.14801/jaitc.2020.10.1.45 · Cited by 16
Multi-granularity sequential neural network for document-level biomedical relation extraction
Liu, X., Tan, K., & Dong, S. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102718 · Cited by 17
PEERAssist: Leveraging on Paper-Review Interactions to Predict Peer Review Decisions
Bharti, P. K., Ranjan, S., Ghosal, T., Agrawal, M., & Ekbal, A. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-91669-5_33 · Cited by 16
SciBERTSUM: Extractive Summarization for Scientific Documents
Sefid, A., & Giles, C. L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-06555-2_46 · Cited by 25
Lu, Q., Dou, D., & Nguyen, T. H. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.325 · Cited by 33
SagDRE: Sequence-Aware Graph-Based Document-Level Relation Extraction with Adaptive Margin Loss
Wei, Y., & Li, Q. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539304 · Cited by 30
“We use SciBERT [1] as the encoder, which is a pretrained language model trained on large-scale labeled scientific corpora.”
MethodInfluentialWang, X.-Q., Xu, X., Tong, W., Liu, Q., & Liu, Z.-C. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.999289 · Cited by 14
Wang, S., Šuster, S., Baldwin, T., & Verspoor, K. M. (2022) · Journal of Medical Internet Research · doi:10.2196/38859 · Cited by 17
“Elkin and Zhu [27] included word-embedding features [36,37] in predicting trial completion but only used static word representations rather than more advanced contextualized word representations derived from pretrained language models [38,39].”
An integrated indicator for evaluating scientific papers: considering academic impact and novelty
Yan, Z., & Fan, K. (2024) · Scientometrics · doi:10.1007/s11192-024-05150-9 · Cited by 18
“To ensure a comprehensive assessment of paper novelty and to further validate our findings, we also introduce SciBERT (Beltagy et al., 2019) and ELMo (Peters et al., 2018), two pre-trained models widely recognized in NLP. SciBERT is designed for NLP tasks in scientific papers, and it provides…”
InfluentialMeasuring the commercial potential of science
Masclans, R., Hasan, S., & Cohen, W. M. (2025) · Strategic Management Journal · doi:10.1002/smj.3720 · Cited by 18
Yang, Y., Cao, Z., Zhao, P., Zeng, D. D., Zhang, Q., & Luo, Y. (2021) · Journal of Safety Science and Resilience · doi:10.1016/j.jnlssr.2021.08.002 · Cited by 16
“SciBERT [36] is pretrained on a corpus of which 82% are biomedical literature and 18% are computer science literature.”
BackgroundRecommending Datasets for Scientific Problem Descriptions
Färber, M., & Leisinger, A.-K. (2021) · International Conference on Information and Knowledge Management · doi:10.1145/3459637.3482166 · Cited by 17
“Following [23], a sample size of 400 is chosen for the user study to ensure the results’ statistical significance with a confidence interval of 95%.”
BackgroundDocFlow: A Visual Analytics System for Question-Based Document Retrieval and Categorization
Qiu, R., Tu, Y., Wang, Y.-S., Yen, P., & Shen, H. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3219762 · Cited by 16
“…large-scale biomedical corpora and has largely outperformed original BERT in a variety of biomedical text mining tasks; while in the scientific area, sciBERT [41] is pretrained on large-scale scientific literature, and demonstrates significant performance improvement on five core scientific NLP tasks.”
BackgroundLeveraging BERTopic for the Analysis of Scientific Papers on Seaweed
Parlina, A., & Maryati, I. (2023) · International Conference on Computer, Control, Informatics and its Applications · doi:10.1109/ic3ina60834.2023.10285737 · Cited by 16
“It achieves this by utilizing unsupervised pretraining on a substantial corpus of scientific publications across various domains, enhancing its effectiveness in subsequent scientific NLP tasks [21].”
BackgroundLi, D., & Zhang, Z. (2023) · PLoS ONE · doi:10.1371/journal.pone.0293034 · Cited by 19
“They may fail to deduce the relationships between columns and rows or to grasp the implications of absent values in the table [34, 35].”
BackgroundArtificial Intelligence in Astronomical Optical Telescopes: Present Status and Future Perspectives
Huang, K., Hu, T., Cai, J., Pan, X.-S., Hou, Y., Xu, L., Wang, H., Zhang, Y., & Cui, X. (2024) · Universe · doi:10.3390/universe10050210 · Cited by 21
Pre-trained language models for keyphrase prediction: A review
Umair, M., Sultana, T., & Lee, Y.-K. (2024) · ICT Express · doi:10.1016/j.icte.2024.05.015 · Cited by 17
“The second subcategory is academic, A number of existing works adopted PLMs [138, 139, 140, 141, 142, 143].”
BackgroundUsing large language model (LLM) to identify high-burden informal caregivers in long-term care
Chien, S.-C., Yen, C.-M., Chang, Y.-H., Chen, Y., Liu, C.-C., Hsiao, Y.-P., Yang, P.-Y., Lin, H.-M., Yang, T.-E., Lu, X., et al. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108329 · Cited by 17
Jin, X., Sun, X., Chen, J., & Sutcliffe, R. F. E. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557318 · Cited by 17
“Asada et al. [25] proposed a SciBERT-based model which fused descriptions and molecular structures of drug entities.”
MethodResultInfluential1.5 million materials narratives generated by chatbots
Park, Y. J., Jerng, S. E., Yoon, S., & Li, J. (2024) · Scientific Data · doi:10.1038/s41597-024-03886-w · Cited by 19
“By using NLP, it is possible to identify a subset of initial material structures with desired characteristics and desired application fields and convert it into actual first-principles calculation input through tools such as pymatgen 26 and ASE 27 .”
MethodKao, W.-T., & Lee, H.-Y. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.189 · Cited by 17
DMDD: A Large-Scale Dataset for Dataset Mentions Detection
Pan, H., Zhang, Q., Dragut, E. C., Caragea, C., & Latecki, L. J. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00592 · Cited by 32
Hooper, R., Goyal, N., Blok, K., & Scholten, L. (2024) · Policy Sciences · doi:10.1007/s11077-024-09548-3 · Cited by 18
“In addition, they can leverage ‘transfer learning’ to enhance portability of the algorithm to a new context (Beltagy et al., 2019 ; Devlin et al., 2018 ; Kyriakakis et al., 2019 ).”
Norouzi, R., Kleinberg, B., Vermunt, J. K., & Lissa, C. J. V. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2024.13 · Cited by 16
“24 Science BERT(SciBERT) 35 : The SciBERT model was pre-trained on a corpus of scientific literature, which makes it particularly well-suited to scientific text mining applications.”
Named Entity Recognition as Structured Span Prediction
Zaratiana, U., Tomeh, N., Holat, P., & Charnois, T. (2022) · doi:10.18653/v1/2022.umios-1.1 · Cited by 27
Miah, M. S. U., Sulaiman, J. B., Sarwar, T. B., Islam, S. S., Rahman, M., & Haque, M. S. (2023) · IEEE EUROCON - International Conference on Computer as a Tool (EUROCON) · doi:10.1109/eurocon56442.2023.10199075 · Cited by 16
“The study compares the performance of several algorithms, including BERT [12], sciBERT [13], DecisionTree [14], RandomForest [15], and Gaussian NaiveBayes [16], on a dataset of published scientific articles.”
MethodSemEval 2022 Task 12: Symlink - Linking Mathematical Symbols to their Descriptions
Lai, V. D., Veyseh, A. P. B., Dernoncourt, F., & Nguyen, T. H. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.230 · Cited by 16
“Then, we used the SciBERT tokenizer (Beltagy et al., 2019) to tokenize the text.”
BackgroundMethodInfluentialDeep learning with language models improves named entity recognition for PharmaCoNER
Sun, C. H., Yang, Z., Wang, L., Zhang, Y., Lin, H., & Wang, J. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04260-y · Cited by 22
“Compared with MultiBERT and Spanish BERT, the best version of SciBERT and BioBERT can obtain higher performance.”
BackgroundInfluentialAutomated Scoring of Translations with BERT Models: Chinese and English Language Case Study
Cui, Y., & Liang, M. (2024) · Applied Sciences · doi:10.3390/app14051925 · Cited by 16
Katzer, B., Klinder, S., & Schulz, K. (2025) · Materials Today Communications · doi:10.1016/j.mtcomm.2025.112186 · Cited by 19
“This procedure leads to several mechanics and materials science LLMs such as MechGPT [20] and ProtAgent [21] or to scientific LLMs in general like SciBERT [22].”
Tyc, J., Selami, T., Hensel, D. S., & Hensel, M. U. (2023) · Architecture · doi:10.3390/architecture3020010 · Cited by 16
“The implementation prepared for this review utilized the scikit-learn implementation of the MultinomialNB algorithm [45] and the SciBERT transformer model [46] for the semi-automated publication screening.”
MethodNer4Opt: Named Entity Recognition for Optimization Modelling from Natural Language
Dakle, P. P., Kadıoğlu, S., Uppuluri, K., Politi, R., Raghavan, P., Rallabandi, S. K., & Srinivasamurthy, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33271-5_20 · Cited by 30
SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation
Kasanishi, T., Isonuma, M., Mori, J., & Sakata, I. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.418 · Cited by 39
Parameter-Efficient Legal Domain Adaptation
Li, J., Bhambhoria, R. V., & Zhu, X. (2022) · Natural Legal Language Processing Workshop (NLLP) · doi:10.18653/v1/2022.nllp-1.10 · Cited by 27
Tekumalla, R., & Banda, J. M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-48044-7_26 · Cited by 22
BioMamba: Domain-Adaptive Biomedical Language Models
Yue, L., Zhu, M., Xing, S., Cao, Y., Wang, Y., Chenthamarakshan, V., Pan, S., Das, P., & Fu, T. (2026) · Health Data Science · doi:10.34133/hds.0454 · Cited by 16
“Transformer-based biomedical models such as BioBERT [2], SciBERT [3], PubMedBERT [4], BioGPT [5], SciFive [6], and BioBART [7] have shown that adapting general-purpose language models to biomedical corpora can yield strong performance.”
Koloveas, P., Chatzopoulos, S., Vergoulis, T., & Tryfonopoulos, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05409-8_13 · Cited by 25
“…features (Jurgens et al., 2018), machine learning methods (Teufel et al., 2006), and, in recent years, on domain-specific pre-trained language models (PLMs), such as SciBERT (Beltagy et al., 2019), that require large scientific datasets such as (Lo et al., 2020) and task-specific architectures 1 .”
InfluentialPatricoski, J., Johns Hopkins Molecular Tumor Board Investigators, Kreimeyer, K., Balan, A., Hardart, K., Tao, J., Anagnostou, V., & Botsis, T. (2022) · Studies in health technology and informatics · doi:10.3233/shti210848 · Cited by 19
“The largest difference between SciBERT and most of the other models is its use of SciVocab, which overlaps with BaseVocab used in the BERT-Base by only 42% [9].”
MethodGonzález‐González, J., de Arriba-Pérez, F., García-Méndez, S., Busto-Castiñeira, A., & González-Castaño, F. J. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.101634 · Cited by 18
“Unfortunately, most pre-trained language models’ embeddings are not adapted to the legal domain (Beltagy et al., 2019), so they tend to underperform compared to traditional ml algorithms like rf , as noted by Chen et al. (2022).”
MethodSciClops: Detecting and Contextualizing Scientific Claims for Assisting Manual Fact-Checking
Smeros, P., Castillo, C., & Aberer, K. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482475 · Cited by 25
“Thus, we introduce three variants of BERT with domain-specific fine-tuning namely, SciBERT , NewsBERT and SciNewsBERT : • SciBERT is pretrained on top of BERT with a corpus from Seman-ticScholar.org containing ~1 𝑀 papers [3].”
MethodAudibert, R. B., Lemos, H., Avelar, P., Tavares, A. R., & Lamb, L. C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.13131 · Cited by 20
Information Retrieval in an Infodemic: The Case of COVID-19 Publications
Teodoro, D., Ferdowsi, S., Borissov, N., Kashani, E., Alvarez, D. V., Copara, J., Gouareb, R., Naderi, N., & Amini, P. (2021) · Journal of Medical Internet Research · doi:10.2196/30161 · Cited by 28
EPICURE: Ensemble Pretrained Models for Extracting Cancer Mutations from Literature
Cao, J., van Veen, E. M., Peek, N. B., Renehan, A. G., & Ananiadou, S. (2021) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms52027.2021.00054 · Cited by 15
Relational distance and document-level contrastive pre-training based relation extraction model
Dong, Y., & Xu, X. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.02.012 · Cited by 16
Leaman, R., Islamaj, R., Adams, V., Alliheedi, M., Almeida, J. R., Antunes, R., Bevan, R., Chang, Y., Erdengasileng, A., Hodgskiss, M., et al. (2023) · Database · doi:10.1093/database/baad005 · Cited by 18
Hunter, S. B., Mathews, F., & Weeds, J. (2023) · Ecological Informatics · doi:10.1016/j.ecoinf.2023.102076 · Cited by 15
Language with vision: A study on grounded word and sentence embeddings
Shahmohammadi, H., Heitmeier, M., Shafaei-Bajestan, E., Lensch, H. P. A., & Baayen, R. H. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02294-z · Cited by 17
“14 https://cs.fit.edu/~mmahoney/compression/textdata.html Our approach, apart from its simplicity, shows excellent generalization, as evidenced by its success on a variety of human-annotated similarity and relatedness tasks, including those involving unseen abstract and concrete words.”
BackgroundChen, Z., Zhang, C., Zhang, H., Zhao, Y., Yang, C., & Yang, Y. (2024) · The Electronic Library · doi:10.1108/el-03-2024-0070 · Cited by 15
Understanding Clinical Trial Reports: Extracting Medical Entities and Their Relations
Nye, B. E., DeYoung, J., Lehman, E., Nenkova, A., Marshall, I. J., & Wallace, B. C. (2020) · arXiv · doi:10.48550/arxiv.2010.03550 · Cited by 15
Zhang, C., & Tian, L. (2023) · Scientometrics · doi:10.1007/s11192-023-04740-3 · Cited by 15
“Next, we visualize the degree of similarity in the usage of research meth-ods across countries and investigate whether there are significant differences in their usage.”
Improving Bug Severity Prediction With Domain-Specific Representation Learning
Wei, Y., Zhang, C., & Ren, T. (2023) · IEEE Access · doi:10.1109/access.2023.3279205 · Cited by 15
Yang, C., Deng, J., Chen, X., & An, Y. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104456 · Cited by 18
ExASAG: Explainable Framework for Automatic Short Answer Grading
Tornqvist, M., Mahamud, M., Guzman, E. M., & Farazouli, A. (2023) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2023.bea-1.29 · Cited by 29
30 years of AI for electrocatalysis: Where we are and what’s next?
Chen, H., Kätelhön, E., Lu, Y., Cheng, J., Tian, Z., & Compton, R. G. (2025) · eScience · doi:10.1016/j.esci.2025.100515 · Cited by 15
Šuster, S., Baldwin, T., & Verspoor, K. M. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.04.006 · Cited by 16
Zhu, C., Xia, X., Li, N., Zhong, F., Yang, Z., & Liu, L. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107262 · Cited by 17
Bautista, Y. J. P., Theran, C., Aló, R., & Lima, V. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-47454-5_17 · Cited by 18
BioBBC: a multi-feature model that enhances the detection of biomedical entities
Alamro, H., Gojobori, T., Essack, M., & Gao, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-58334-x · Cited by 17
Otegi, A., Campos, J. A., Azkune, G., Soroa, A., & Agirre, E. (2020) · Workshop on NLP for COVID-19 (NLP-COVID19) · doi:10.18653/v1/2020.nlpcovid19-2.15 · Cited by 15
Dobbins, N. J., Mullen, T., Uzuner, Ö., & Yetişgen, M. (2022) · Scientific Data · doi:10.1038/s41597-022-01521-0 · Cited by 15
“We first trained NER models with SciBERT using the manually annotated portion and tested on the semi-automated portion, then reversed the experiment and trained on the semi-automated portion and tested on the manually annotated portion.”
MethodInfluentialKumar, P., Kabra, S., & Cole, J. M. (2024) · Scientific Data · doi:10.1038/s41597-024-03979-6 · Cited by 18
Exploring developments of the AI field from the perspective of methods, datasets, and metrics
Yao, R., Ye, Y., Zhang, J., Li, S., & Wu, O. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103157 · Cited by 16
LITERAS: Biomedical literature review and citation retrieval agents
Gorenshtein, A., Shihada, K., Sorka, M., Aran, D., & Shelly, S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110363 · Cited by 18
Requirements Analysis for an Open Research Knowledge Graph
Brack, A., Hoppe, A., Stocker, M., Auer, S., & Ewerth, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-54956-5_1 · Cited by 15
CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model
Chiang, S.-H., Wang, S.-C., & Fan, Y.-C. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.429 · Cited by 15
“Note SciBERT which is pre-trained based on science Models P@1 F1@3 MRR@10 NDCG@10 DGen (Wordnet CSG) 9.31”
InfluentialDomain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature
Wang, Y., Li, J., Naumann, T., Xiong, C., Cheng, H., Tinn, R., Wong, C., Usuyama, N., Rogahn, R., Shen, Z., et al. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3469053 · Cited by 20
Sent2Span: Span Detection for PICO Extraction in the Biomedical Text without Span Annotations
Liu, S., Sun, Y., Li, B., Wang, W., Bourgeois, F. T., & Dunn, A. G. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.147 · Cited by 12
Flexible Instance-Specific Rationalization of NLP Models
Chrysostomou, G., & Αλέτρας, Ν. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21298 · Cited by 24
Gupta, K. K., Barman, S., Sankar, A., Dey, S., & Mukhopadhyay, T. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-025-10486-3 · Cited by 16
“Gupta et al. [136 ] finetuned SciBERT [ 152] with the corpus of the published literature on families of materials such as inorganic glasses, metallic glasses, alloys, etc. and developed MatSciBERT.”
JurisBERT: A New Approach that Converts a Classification Corpus into an STS One
Viegas, C. F. O., Costa, B. C., & Ishii, R. P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36805-9_24 · Cited by 18
Better entity matching with transformers through ensembles
Low, J. F., Fung, B. C. M., & Xiong, P. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111678 · Cited by 15
Rahman, F. I., Halim, S. M., Singhal, A., & Khan, L. R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-65172-4_13 · Cited by 16
OSTIS: A novel Organization-Specific Threat Intelligence System
Arikkat, D. R., Vinod, P., A., R. R. K., Nicolazzo, S., Nocera, A., Timpau, G., & Conti, M. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103990 · Cited by 15
Using large language models for semantic interoperability: A systematic literature review
Abu-Salih, B., Alotaibi, S., Alanazi, A. L., Abukhurma, R., Al-Shboul, B. A., Khoury, A., & Aljaafari, M. A. H. (2025) · ICT Express · doi:10.1016/j.icte.2025.06.011 · Cited by 18
Automatic Analysis of Substantiation in Scientific Peer Reviews
Guo, Y., Shang, G., Rennard, V., Vazirgiannis, M., & Clavel, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.684 · Cited by 34
Using BERT to Identify Causal Structure in Students’ Scientific Explanations
Cochran, K., Cohn, C., Hastings, P., Tomuro, N., & Hughes, S. (2023) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00373-y · Cited by 18
“…further pre-training of BERT or large language models (i.e. ‘fine-tuning of a language model’ (Chakrabarty et al., 2019) ) using domain corpora have reported improved performance (Gururangan et al., 2020; Alsentzer et al., 2019; Sung et al., 2019; Lee et al., 2020; Beltagy et al., 2019; Cohn, 2020).”
Mindful Explanations: Prevalence and Impact of Mind Attribution in XAI Research
Hindennach, S., Shi, L., Miletić, F., & Bulling, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641009 · Cited by 17
“We used SciBERT [4], a BERT model trained on scientific texts, to produce contextualised word embeddings.”
Method
Metadata
- Authors
- Beltagy, I., Lo, K., & Cohan, A.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- ORCID
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 3613-3618
- OpenAlex
- W2970771982
- Semantic Scholar
- 156d217b0a911af97fa1b5a71dc909ccef7a8028
- Cited by
- 3,218
- References
- 33
- Citations by year
- 2026: 328 · 2025: 534 · 2024: 542 · 2023: 626 · 2022: 478 · 2021: 486 · 2020: 203 · 2019: 19
- FWCI
- 113.46
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- SciBERT, pre-trained language models, scientific texts, scientific publications, biomedical text, natural language processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.