Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Wenpeng Yin, Jamaal Hay, Dan Roth. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is evidenced by LLMs impressive zero-shot and few-shot learning ability [18,132], as well as the fact that in-context learning does not require weight update, which mechanically prevents unfavorable fine-tuning-related behavior like catastrophic forgetting.”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [132]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have been shown to outperform previous zero-shot methods [132].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [132]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 6×
“Regarding this label-fully-unseen evaluation, apart from our entailment models and three unsupervised baselines “Majority”, “Word2Vec” and “ESA”, we also report the following baseline: Wikipedia-based : We train a binary clas-sifier based on BERT on a dataset collected from Wikipedia.”
MethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 6×
“Regarding this label-fully-unseen evaluation, apart from our entailment models and three unsupervised baselines “Majority”, “Word2Vec” and “ESA”, we also report the following baseline: Wikipedia-based : We train a binary clas-sifier based on BERT on a dataset collected from Wikipedia.”
MethodInfluentialDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 2×
“• Word2Vec 5 (Mikolov et al., 2013): Both the representations of the text and the labels are the addition of word embeddings element-wisely.”
BackgroundGLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · cited 11×
“For label-partially-unseen setup in which we intentionally provide annotated data, we first pretrain BERT on the MNLI/FEVER/RTE, then fine-tune on the provided training data.”
BackgroundMethodInfluentialA Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · cited 2×
“, 2019), and train it on three mainstream entailment datasets: MNLI (Williams et al., 2018), GLUE RTE (Dagan et al.”
BackgroundMethodCharacter-level Convolutional Networks for Text Classification
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 3,205 · cited 1×
“We use the large-scale Yahoo dataset released by Zhang et al. (2015).”
MethodComputing semantic relatedness using Wikipedia-based explicit semantic analysis
Gabrilovich, E., & Markovitch, S. (2007) · Cited by 1,981
The PASCAL Recognising Textual Entailment Challenge
Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture Notes in Computer Science · doi:10.1007/11736790_9 · Cited by 1,628
(2018) · International Conference on Learning Representations · Cited by 1,621
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343 · cited 1×
“In this work, we make use of the widely-recognized state of the art entailment technique – BERT (Devlin et al., 2019), and train it on three mainstream entailment datasets: MNLI (Williams et al., 2018), GLUE RTE (Dagan et al., 2005; Wang et al., 2019) and FEVER 3 (Thorne et al., 2018), respectively.”
Method
Research works printed but not linked: 2
- [8]8. Chenliang Li, Jian Xing, Aixin Sun, and Zongyang Ma. 2016a. Effective document labeling with very few seed words: A topic model approach. In CIKM, pages 85-94.
- [11]11. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed repre- sentations of words and phrases and their composi- tionality. In NIPS, pages 3111-3119.
Cited by
Items 801–867 of 867
Probing large language models for autonomous driving behavior
Bao, Z., Zhao, W., & Li, Q. (2026) · Journal of Intelligent and Connected Vehicles · doi:10.26599/jicv.2026.9210095
Cargnelutti, M., Brobston, C., English, E., Sadow, J., Bailey, K., Leppert, G., Watson, A., Chapel, J., & Zittrain, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.18972
Scarce data and complex texts: the role of LLMs in automatic text classification
Palmieri, E. <. (2026) · AMS Dottorato Institutional Doctoral Theses Repository (University of Bologna) · doi:10.48676/unibo/amsdottorato/12674
CzechTopic: A Benchmark for Zero-Shot Topic Localization in Historical Czech Documents
Kostelník, M., Hradiš, M., & Dočekal, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36033-5_27
Determinants of Starting Salaries for Filipino Graduates: An Explainable Machine Learning ApproachPreprint
Aranes, A. G. A., Magpantay, J. M. C., Recario, R. N. C., Cruz, J. I. N. G., & Camaclang, R. C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.21383
“Recent work extends these with Natural Language Processing, where reformulating classification as a Natural Language Inference (NLI) task, that is, scoring whether a text premise entails a class hypothesis, enables effective prediction in low-resource domains [18, 7].”
TCMB para politikası iletişimindeki geçişlerin doğal dil işleme ve yapay zekâ temelinde analizi
Görgün, M. F., Özgüler, İ. C., & Demireli, E. (2026) · Erciyes Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi · doi:10.18070/erciyesiibd.1861563
VeriCam: A Verification Baseline for the Classification of Unknown DataPreprint
Wojcik, L., Lima, G. E., Silva, S. M., Nascimento, E., & Menotti, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.31107
“Some recent works extend OOD towards zero-shot classification [4], aiming to categorize unseen data in new classes, effectively tackling the multi-unknown class issue.”
A systematic Approach to constructing a Chance-and-Risk Matrix for Semiconductor Supply ChainsPreprint
Salkić, E., Fichtl, A. M., Ulrich, P., Ehm, H., Bonik, M., & Groh, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.01563
“Recent advances in LLMs have made in-context learning (Brown et al. 2020) viable, enabling zero-shot classification (Yin et al. 2019) and structured extraction via taxonomy-guided prompts without task-specific training.”
Li, W., Barata, C., Constante, M., & Sanches, J. (2026) · IEEE Access · doi:10.1109/access.2026.3750234
Wu, S., & Tsai, T. H. (2023) · Institutional Repositories DataBase (IRDB) · doi:10.20736/0002001306
“Zero-Shot Prompting is a novel technique in deep learning prompt engineering that aims to generate natural language outputs without requiring any labeled data or fine-tuning [4][5].”
Jallad, K. A., Ghneim, N., & Rebdawi, G. (2026) · IEEE Access · doi:10.1109/access.2026.3732060
“Some state-of-the-art models that contain Arabic language are: Facebook Bart Large MNLI [32], [33], Face-book RoBERTa-Large-MNLI [16], Microsoft Multilingual MiniLM [34], Microsoft XLM-RoBERTa [34], Microsoft DeBERTaV3 [35].”
Shelat, S., Salvi, S., Roy, S., Gaur, M., & Sheth, A. (2026) · arXiv (Cornell University) · doi:10.13016/m2yszk-yg5b
An AI-Based Approach for Analysing Dependencies Among User Stories
Hofmann, L. S., Lauer, A., Kosiol, J., Wieber, P., Braun, D., & Taentzer, G. (2026) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re68928.2026.00023
“For the transformer-based approach, we use BART-Large-MNLI [32], [33] FortheLLM-based approach, we use the proprietary gpt-5.”
Peshavariya, B., & Nahar, S. (2026) · Informatica · doi:10.31449/inf.v50i15.14996
Quantifying Sustainability for Structured Credit Products: An NLP-Driven ESG Scoring ApproachPreprint
Bhattacharya, B., & Medda, F. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7439699
da Silva, L. F. (2026) · Pedro e João Editores eBooks · doi:10.51795/9786526531730
Integrasi Model Zero-Shot Classification Pada Sistem Tata Kelola Produk Toko Online Alfamidi
Amalia, S., & Suendri, S. (2026) · Jurnal Komputer dan Elektro Sains · doi:10.58291/komets.v4i2.699
Afreen, N., Boratto, L., Fenu, G., Malloci, F. M., Marras, M., & Soccol, A. (2026) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-026-01261-4
“We employed a zero-shot classifier (Yin et al. 2019 ) to extract occupational groups based on ESCO 4 profession labels (e.g., Science and engineering associate professionals and Information and communications technology professionals ) from audiences and goals texts.”
Representational Learning for Grounded Reasoning Across Modalities
Li, B. (2026) · eScholarship (California Digital Library)
Mapping Palm Line Morphology to Cognitive Styles via Zero-Shot Sentence Embeddings
Das, A., Mondal, A., Yadav, V., & Das, S. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-14775-2_12
Reflexionen zum Einsatz künstlicher Intelligenz in der angewandten Wirtschaftsgeographie
Losacker, S., & Schütz, M. (2026) · Standort · doi:10.1007/s00548-026-01069-w
“Textklassifizierungsmodelle sind in der Lage, Texte ent-lang bestimmter Muster oder Kategorien zu ordnen (siehe z. B. Devlin et al. 2019; Yin et al. 2019 oder Laurer et al. 2024).”
Demirkiran, E. T. (2026) · Research Square · doi:10.21203/rs.3.rs-10975833/v1
Custom Named Entity Recognition and Topic Classification for Global Health PublicationsPreprint
Skura, G., Geissbühler, A., & Falcone, J.-L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.24625
Ahmed, Z., Qin, Y., & Huang, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.22094
“…al. (2025) demonstrates scalable few-shot moderation using LLMs. Chain-of-thought (CoT) prompting (Wei et al., 2022) where the content serves as a premise and the policy description as a hypothesis (Yin et al., 2019; Calabrese et al., 2022), naturally supporting zero-shot transfer to new policies.”
Attribute-guided latent space alignment for zero-shot text classification of hotline requests
Fan, Q., Diao, Y., & Han, H. (2026) · Machine Learning with Applications · doi:10.1016/j.mlwa.2026.101025
Fredyan, R., Salim, L. F., & Khairunnisa (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.08.339
Frøsig, T. B., & Roméro, M. (2026) · Learning Culture and Social Interaction · doi:10.1016/j.lcsi.2026.101065
Wright, A. T., Liu, S., & Wright, A. P. (2026) · medRxiv · doi:10.64898/2026.09.24.26363906
Typic: A Compact Open Model for Typed, Calibrated DecisionsPreprint
Shovon, M. M. (2026) · Research Square · doi:10.21203/rs.3.rs-11162889/v1
Jallad, K. A., Ghneim, N., & Rebdawi, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33530
“Some state-of-the-art models in English are: PaLM 540B[19], Vega_v2_6B[20], RoBERTa[21], SemBERT[22], XLNET[23], AlexaTM_20B[24] , BloombergGPT and GPT-NeoX and BLOOM_176B [25], UnitedSynT5[26], ByT5[27] , Rethinking Coupling[28], mGPT[29], DeBERTa[30], SpanBERT[31] , SqueezeBERT[32] ,DistilBERT[33].”
InfluentialAn NLP-Based Chatbot System for Automated Grievance Classification and Management
Saju, B., T, G. K., Sulaiman, A., Krishnan, V. A. H., & Amrith Nath Murali, T. (2026) · International Conference on Innovations in Mechanical, Robotics, Computing and Biomedical Engineering (ICIMRBE) · doi:10.1109/icimrbe70745.2026.11709984
“Yin et al. [12] did zero-shot text classification and showed the ability of pretrained transformer models to classify documents into unseen categories using natural language inference.”
Tung-lersloy, C., Paireekreng, W., & Prinyapol, N. (2025) · Journal of Engineering and Digital Technology (JEDT)
Juarto, B., Tiara, K. A., Rahmania, R., & Chen, V. F. (2026) · International Conference on ICT for Smart Society (ICISS) · doi:10.1109/iciss71466.2026.11715381
“A hybrid extension could run an entailment classifier [28] or instruction-tuned guard [26], [27] alongside the lexicon.”
Text-Based Policy Signal Analysis of Crude Petroleum Export Revenue Movements Across APEC Economies
Gunawan, J. C., Aurell, H. C., & Saputra, D. (2026) · International Conference on ICT for Smart Society (ICISS) · doi:10.1109/iciss71466.2026.11715424
“The dominant policy signal is determined by the highest similarity score among the predefined policy categories [24], [25].”
Arachchige, I. M. D. M. (2025) · Adelaide Research & Scholarship (AR&S) (University of Adelaide)
Divya N, D. V. J. C. (2025) · International Journal of Innovation Studies
Shrome at Touché: Soft-Vote Ensembling and Counter-Causal Augmentation for Causality ExtractionPreprint
Nobari, R. Z., & Sooratgar, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.03268
“Yin et al. [16] showed that off-the-shelf natural-language-inference models can act as zero-shot text classifiers by phrasing each candidate label as an entailment hypothesis, and the same paradigm has since been used to verify generated or extracted text.”
Automated structuring of text space with minimal supervision
Zhang, Y. (2025) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Akash, P. S. (2025) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Efficient and effective learning of text representations
Meng, Y. (2023) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Towards open-domain event understanding
Li, S. (2024) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Situated Natural Language Processing: Perspectivist Methods for Modelling and Evaluation
LO, S. M. (n.d.) · Institutional Research Information System University of Turin (University of Turin)
Merlet, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7588739
Models and data for behavioural spreading in social networks
Baratto, M. (n.d.) · Institutional Research Information System University of Turin (University of Turin)
Merlet, A. (2026) · Research Square · doi:10.21203/rs.3.rs-11317705/v1
ZeroLINC: Training-Free Local Classification of Security Incident Reports
Kapelinski, C., Machado, B., & Kreutz, D. (2026) · Simpósio Brasileiro de Cibersegurança (SBSeg) · doi:10.5753/sbseg_estendido.2026.33502
“A Natural Language Inference (NLI) model judges whether one sentence follows from another; classification becomes asking, for each category, whether the ticket supports the statement “this report describes ⟨ category ⟩ ” [Yin et al. 2019].”
InfluentialAgriJev: Efficient Agricultural Candidate Scoring with Multi-Scale Adapted ModelsPreprint
Li, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.19478
“Entail-ment classification represents candidate descriptions as hypotheses (Yin et al., 2019; Laurer et al., 2023).”
Cognitive Bias in News: Implications for Sentiment, Emotion, and Systemic Risk
Stander, Y. S. (2026) · Econometrics · doi:10.3390/econometrics14040053
EarlyCommit: Calibrated Word-by-Word Commitment for Robots Acting Before You Finish SpeakingPreprint
Khan, M. H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.20531
“Options are defined per request, in the spirit of zero-shot label scoring [21].”
Audio System One Model: Typed Probabilistic Perception of Emotion in SpeechPreprint
Bellver-Soler, J., Martín-Fernández, I., Guragain, A., & D’Haro, L. F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.22401
“…much older than its current name: dataless classification embedded label names [17], entailment classifiers test each label as a yes/no hypothesis [18], encoders score all the written options of a question in one pass [19] or several label sets at once [20], and dual encoders carried the idea to…”
EpiLens: Lightweight RAG for Epidemiological Knowledge AcquisitionPreprint
Perri, V., Cattuto, C., & Paolotti, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.26730
A Compact Open Encoder Model for Calibrated Typed Decisions on Unseen TasksPreprint
Shovon, M. M. (2026) · Research Square · doi:10.21203/rs.3.rs-11163029/v1
The edges of truth: A study of evaluating how good explainable networks are at classifying newsPreprint
Herczeg, A., & Dînșoreanu, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7655828
Cheimariotis, I., Costa, J. N., Piazza, F., Lodi, C., Rataj, M., Zawieska, J., Stępniak, M., & Grosso, M. (2026) · European Transport Research Review · doi:10.1186/s12544-026-00854-w
“…stance, topic and frame detection and on political-science text classification [11, 12]; zero-shot, few-shot, fine-tuned and instruction-tuned approaches have been compared systematically [13, 14]; and data-efficient alternatives based on entailment and transfer learning have been proposed [15–17].”
The Hitchhiker's Guide to System 1 Models: Fast Decision-Making Beyond LLMSPreprint
Roitman, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.33775
Çataltaş, M., Arslan, A., & Baykan, N. A. (2026) · Sakarya University Journal of Computer and Information Sciences · doi:10.35377/saucis...1898109
Mosaic: Automated Intent Description Generation for Multi-Label Classification
Chatterjee, A., Crestan, E., Saha, R., Mordoch, R., & Acharyya, S. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-industry.33
MulEG: Multi-Level Explanation-Guided Feature Attribute Graph for Zero-Shot Tabular Classification
Zhang, J., Zhao, Y., Kang, X., Yan, Z., Ma, B., Chai, Q., Yu, X., & Li, R. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.2301
Bardhan, R., Sumanth, V., Dorai, G., Regmi, P., & Kumar, P. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8110359
LLMs Propose, Verifiers Decide: Selective Intent Classification for Short Web Queries
Bai, Y., & Luo, R. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3839871
Horizontal emotional pathways: A path-dependent perspective on guest evaluation in online reviews
Park, J. (2026) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2026.104962
Da, N. T., & Vo, T.-D. (2026) · SN Business & Economics · doi:10.1007/s43546-026-01536-4
Entre Nota e Opinião: Análise de Sentimento em Reviews para Recomendação Colaborativa
Vasconcelos, N., Andrade, Y., Tuler, E., Dur�ão, F. A., Pereira, A. C. M., & Rocha, L. (2026) · Brazilian Symposium on Multimedia and the Web · doi:10.5753/webmedia.2026.30191
Evaluating Jev on Arabic: Comparing Decision Models and LLMs Across Arabic Tasks and DialectsPreprint
Elsafty, H., Deußer, T., Sparrenberg, L., Ramamurthy, R., & Sifa, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.10802
النعاس, و. ع. م. (2026) · Shihab Journal of Humanities · doi:10.65405/sjh.2.3.266
HandsOnAid: Building and Evaluating an Offline First-Aid Assistant for Emergency Response ScenariosPreprint
Salvi, R. I., Muna, S. S., & Mushfique, M. R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.14059
Metadata
- Authors
- Yin, W., Hay, J., & Roth, D.
- Affiliations
- University of Pennsylvania (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 3912-3921
- OpenAlex
- W2970200208
- Cited by
- 537
- References
- 30
- Citations by year
- 2026: 66 · 2025: 86 · 2024: 75 · 2023: 141 · 2022: 94 · 2021: 64 · 2020: 11
- FWCI
- 16.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- U.S. Department of Defense
- Defense Advanced Research Projects Agency (HR0011-15-C-0113)
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- zero-shot text classification, benchmarking, evaluation metrics, logical consequence, cross-domain classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

