The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Neural Network Acceptability Judgments
Open accessAbstract
This paper investigates the ability of artificial neural networks to judge the grammatical acceptability of a sentence, with the goal of testing their linguistic competence. We introduce the Corpus of Linguistic Acceptability (CoLA), a set of 10,657 English sentences labeled as grammatical or ungrammatical from published linguistics literature. As baselines, we train several recurrent neural network models on acceptability classification, and find that our models outperform unsupervised models by Lau et al. (2016) on CoLA. Error-analysis on specific grammatical phenomena reveals that both Lau et al.’s models and ours learn systematic generalizations like subject-verb-object order. However, all models we test perform far below human level on a wide range of grammatical constructions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
With a large enough number of experts and sufficient activation sparsity, experts in mixture-of-experts models may become individually interpretable.
“Several approaches, such as 'mixture of experts' (Shazeer et al., 2017;Fedus et al., 2022;He, 2024), use sparsely activating components -with a large enough number of experts, and with sufficient activation sparsity, experts may become individually interpretable (Warstadt et al., 2019).”
From Open Problems in Mechanistic Interpretability (2025) · cites this paper as Warstadt et al., 2019
The Experiments and Discussion sections address neural-network models for sentence acceptability judgments, not mixture-of-experts models. Searches for activation sparsity and individually interpretable experts found no passage addressing the attributed claim; our copy therefore neither supports nor contradicts it. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ASPECTS OF THE THEORY OF SYNTAX
Chomsky, N. (1964) · doi:10.21236/ad0616323 · Cited by 13,587
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317
Targeted Syntactic Evaluation of Language Models
Marvin, R., & Linzen, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1151 · Cited by 334
Flagged first, then by citations
Research works printed but not linked: 57
- Adger David, 2003, Core Syntax: A Minimalist Approach, 10.1093/oso/9780199243709.001.0001 · DOI
- Adi Yossi, 2017, Proceedings of ICLR Conference Track
- Baltin Mark R., 1982, Linguistic Inquiry, 13, 1
- Baltin Mark R., 2001, Handbook of Contemporary Syntactic Theory, 10.1002/9780470756416 · DOI
- Berwick Robert C., 2011, Cognitive Science, 35, 1207, 10.1111/j.1551-6709.2011.01189.x · DOI
- Bird Steven, 2004, Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions, 31, 10.3115/1219044.1219075 · DOI
- Bresnan Joan W., 1973, Linguistic Inquiry, 4, 275
- Carnie Andrew, 2013, Syntax: A Generative Introduction
- Chomsky Noam, 1957, Syntactic Structures, 10.1515/9783112316009 · DOI · Paper page
- Chomsky Noam, 1995, The Minimalist Program
- Chung Sandra, 1995, Natural Language Semantics, 3, 239, 10.1007/BF01248819 · DOI
- Clark Alexander, 2011, Linguistic Nativism and the Poverty of the Stimulus, 10.1002/9781444390568 · DOI
- Collins Chris, 2005, Syntax, 8, 81, 10.1111/j.1467-9612.2005.00076.x · DOI
- Conneau Alexis, 2017, Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 670
- Conneau Alexis, 2018, ACL 2018-56th Annual Meeting of the Association for Computational Linguistics, 1, 2126
- 10.1162/002438999554200 · DOI
- Dayal Veneeta, 1998, Linguistics and Philosophy, 21, 433, 10.1023/A:1005494000753 · DOI
- Ettinger Allyson, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 1790
- Ettinger Allyson, 2016, Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, 134, 10.18653/v1/W16-2524 · DOI
- Futrell Richard, 2018, arXiv preprint arXiv:1809.01329 · arXiv
- Gazdar Gerald, 1981, The Formal Complexity of Natural Language, 183, 10.1007/978-94-009-3401-6_8 · DOI
- Gibson Edward, 2010, Trends in Cognitive Sciences, 14, 233, 10.1016/j.tics.2010.03.005 · DOI
- Goldberg Adele E., 2004, Language, 80, 532, 10.1353/lan.2004.0129 · DOI
- Hart Betty, 1992, Developmental Psychology, 28, 1096, 10.1037/0012-1649.28.6.1096 · DOI
- Heilman Michael, 2014, Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2, 174, 10.3115/v1/P14-2029 · DOI
- Jackendoff Ray S., 1971, Linguistic Inquiry, 2, 21
- Kadmon Nirit, 1993, Linguistics and Philosophy, 16, 353, 10.1007/BF00985272 · DOI
- Kim Jong-Bok, 2008, English Syntax: An Introduction
- Kingma Diederik P., 2014, Proceedings of the 3rd International Conference on Learning Representations
- Kiros Ryan, 2015, Advances in Neural Information Processing Systems, 3294
- Lau Jey Han, 2016, Cognitive Science, 41, 1202
- Lawrence Steve, 2000, IEEE Transactions on Knowledge and Data Engineering, 12, 126, 10.1109/69.842255 · DOI
- Levin Beth, 1993, English Verb Classes and Alternations: A preliminary investigation
- 10.1162/tacl_a_00115 · DOI
- Linzen Tal, 2018, Glossa: A Journal of General Linguistics, 3, 100, 10.5334/gjgl.528 · DOI
- Mahowald Kyle, 2016, Language, 92, 619, 10.1353/lan.2016.0052 · DOI
- Miller Jim, 2002, An Introduction to English Syntax
- Pennington Jeffrey, 2014, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 1532
- Peters Matthew, 2018, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 1, 2227
- Pullum Geoffrey K., 2002, Linguistic Review, 18, 9
- Hovav Malka Rappaport, 2008, Journal of Linguistics, 44, 129, 10.1017/S0022226707004975 · DOI
- John Robert Ross. 1967. Constraints on Variables in Syntax. Ph.D. thesis, MIT.
- Sag Ivan A., 1997, Journal of Linguistics, 33, 431, 10.1017/S002222679700652X · DOI
- Sag Ivan A., 1985, Natural Language & Linguistic Theory, 3, 117, 10.1007/BF00133839 · DOI
- Sag Ivan A., 2003, Syntactic Theory: A Formal Introduction
- Schütze Carson T., 1996, The Empirical Base of Linguistics: Grammaticality Judgments and Linguistic Methodology
- Shi Xing, 2016, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1526
- Sportiche Dominique, 2013, An Introduction to Syntactic Analysis and Theory
- Sprouse Jon, 2012, Journal of Linguistics, 48, 609, 10.1017/S0022226712000011 · DOI
- Sprouse Jon, 2017, Behavioral and Brain Sciences, 40, e311, 10.1017/S0140525X17000590 · DOI
- Sprouse Jon, 2013, Lingua, 134, 219, 10.1016/j.lingua.2013.07.002 · DOI
- Wagner Joachim, 2009, CALICO Journal, 26, 474, 10.1558/cj.v26i3.474-490 · DOI
- Wang Alex, 2018, Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 353, 10.18653/v1/W18-5446 · DOI · Paper page
- Warstadt Alex, 2019, arXiv preprint arXiv:1901.03438 · arXiv
- Wilcox Ethan, 2018, Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 211, 10.18653/v1/W18-5423 · DOI
- Wilcox Ethan, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 3302
- Williams Edwin, 1980, Linguistic Inquiry, 11, 203
Cited by
Sorted by citations
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Self-citationLoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020) · arXiv · doi:10.48550/arxiv.2003.10555 · Cited by 1,844
SpanBERT: Improving Pre-training by Representing and Predicting Spans
Joshi, M., Chen, D., Liu, Y., Weld, D. S., Zettlemoyer, L., & Levy, O. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00300 · Cited by 1,827
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482
Making Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380
Unified Language Model Pre-training for Natural Language Understanding and Generation
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W. (2019) · arXiv · doi:10.48550/arxiv.1905.03197 · Cited by 1,225
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
Ben Zaken, E., Goldberg, Y., & Ravfogel, S. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.1 · Cited by 765
ChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758
“981 2 282918 /66664 3 CoLa S EN Linguisticacceptability No Binary classification CoLA [64] / [65] 3 Yes 1042 1042 0 0 (0”
BackgroundMobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Sun, Z., Yu, H., Song, X., Liu, R., Yang, Y., & Zhou, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.195 · Cited by 757
Targeted Syntactic Evaluation of Language Models
Marvin, R., & Linzen, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1151 · Cited by 334
Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.05482 · Cited by 307
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., & Gardner, M. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.98 · Cited by 306
Open Problems in Mechanistic Interpretability
Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.16496 · Cited by 291
WARP: Word-level Adversarial ReProgramming
Hambardzumyan, K., Khachatrian, H., & May, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.381 · Cited by 243
Muppet: Massive Multi-task Representations with Pre-Finetuning
Aghajanyan, A., Gupta, A., Shrivastava, A., Chen, X., Zettlemoyer, L., & Gupta, S. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.468 · Cited by 183
MetaICL: Learning to Learn In Context
Min, S., Lewis, M., Zettlemoyer, L., & Hajishirzi, H. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.201 · Cited by 169
Legalbench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
Guha, N., Nyarko, J., Ho, D. E., Ré, C., Chilton, A., Narayana, A., Chohlas-Wood, A., Peters, A., Waldon, B., Rockmore, D., et al. (2023) · doi:10.2139/ssrn.4583531 · Cited by 154
A Practical Survey on Faster and Lighter Transformers
Fournier, Q., Caron, G. M., & Aloise, D. (2023) · ACM Computing Surveys · doi:10.1145/3586074 · Cited by 145
Structured Pruning Learns Compact and Accurate Models
Xia, M., Zhong, Z., & Chen, D. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.107 · Cited by 136
A Systematic Assessment of Syntactic Generalization in Neural Language Models
Hu, J., Gauthier, J., Qian, P., Wilcox, E., & Levy, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.158 · Cited by 134
SqueezeBERT: What can computer vision teach NLP about efficient neural networks?
Iandola, F., Shaw, A., Krishna, R., & Keutzer, K. (2020) · Proceedings of SustaiNLP: Workshop on Simple and Efficient Natural Language Processing · doi:10.18653/v1/2020.sustainlp-1.17 · Cited by 114
Demystifying Prompts in Language Models via Perplexity Estimation
Gonen, H., Iyer, S., Blevins, T., Smith, N., & Zettlemoyer, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.679 · Cited by 109
Chung, J., Kamar, E., & Amershi, S. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.34 · Cited by 72
The MultiBERTs: BERT Reproductions for Robustness Analysis
Sellam, T., Yadlowsky, S., Wei, J., Saphra, N., D'Amour, A., Linzen, T., Bastings, J., Turc, I., Eisenstein, J., Das, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.16163 · Cited by 37
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
Burns, C., Izmailov, P., Kirchner, J. H., Baker, B., Gao, L., Aschenbrenner, L., Chen, Y., Ecoffet, A., Joglekar, M., Leike, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.09390 · Cited by 34
BLiMP: the benchmark of linguistic minimal pairs for English
Warstadt, A., Parrish, A., Liu, H., Mohananey, A., Peng, W., Wang, S.-F., & Bowman, S. R. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00321 · Cited by 28
Self-citationDelta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models
Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.06904 · Cited by 20
Gender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Models
Ghanbarzadeh, S., Huang, Y., Palangi, H., Cruz Moreno, R., & Khanpour, H. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.336 · Cited by 17
Trained on 100 million words and still in shape: BERT meets British National Corpus
Samuel, D., Kutuzov, A., Øvrelid, L., & Velldal, E. (2023) · Findings of the Association for Computational Linguistics: EACL 2023 · doi:10.18653/v1/2023.findings-eacl.146 · Cited by 14
Learning Interpretable Style Embeddings via Prompting LLMs
Patel, A., Rao, D., Kothary, A., McKeown, K., & Callison-Burch, C. (2023) · arXiv · doi:10.48550/arxiv.2305.12696 · Cited by 10
Large language models robustness against perturbation
Alahmari, S. S., Hall, L., Mouton, P. R., & Goldgof, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-29770-0 · Cited by 5
Anchor Points: Benchmarking Models with Much Fewer Examples
Vivek, R., Ethayarajh, K., Yang, D., & Kiela, D. (2024) · Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.eacl-long.95 · Cited by 4
Anchor Points: Benchmarking Models with Much Fewer Examples
Vivek, R., Ethayarajh, K., Yang, D., & Kiela, D. (2023) · arXiv · doi:10.48550/arxiv.2309.08638 · Cited by 4
AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
Fang, J., Jiang, H., Wang, K., Ma, Y., Jie, S., Wang, X., He, X., & Chua, T.-S. (2024) · arXiv · doi:10.48550/arxiv.2410.02355 · Cited by 170
“CoLA (Corpus of Linguistic Acceptability) (Warstadt et al., 2019) is a single-sentence classification task, where sentences from linguistic theory books and journals are annotated as either grammatically acceptable or unacceptable.”
Pretraining Language Models Using Translationese
Doshi, M., Dabre, R., & Bhattacharyya, P. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.334 · Cited by 1
Metadata
- Authors
- Warstadt, A., Singh, A., & Bowman, S. R.
- Affiliations
- New York University (US)
- Meta (United States) (US)
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2019-11
- Volume
- 7
- Pages
- 625-641
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2806120502
- Cited by
- 1,215
- References
- 7
- Citations by year
- 2026: 86 · 2025: 131 · 2024: 160 · 2023: 231 · 2022: 181 · 2021: 224 · 2020: 134 · 2019: 61
- FWCI
- 60.37
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1850208)
- Tencent
- Samsung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Keywords
- artificial neural networks, recurrent neural networks, language skills, error analysis, subject-verb-object order, systematic generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
