Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
Open accessThe DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
A machine learning system can score well on a given test set by relying on heuristics that are effective for frequent example types but break down in more challenging cases.We study this issue within natural language inference (NLI), the task of determining whether one sentence entails another.We hypothesize that statistical NLI models may adopt three fallible syntactic heuristics: the lexical overlap heuristic, the subsequence heuristic, and the constituent heuristic.To determine whether models have adopted these heuristics, we introduce a controlled evaluation set called HANS (Heuristic Analysis for NLI Systems), which contains many examples where the heuristics fail.We find that models trained on MNLI, including BERT, a state-of-the-art model, perform very poorly on HANS, suggesting that they have indeed adopted these heuristics.We conclude that there is substantial room for improvement in NLI systems, and that the HANS dataset can motivate and measure progress in this area.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“For instance, McCoy et al. [61] fnd that state-of-the-art models in natural language inference adopt three fallible syntactic heuristics and perform around random chance when tested on examples where these heuristics fail.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [61]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The second trend is that neural networks routinely fail on exactly the phenomena where such symbolic models succeed, for example, logical inference [6,9], compositionality [7,23] and out-of-distribution generalization [24–26].”
From Symbols and grounding in large language models (2023) · cites this paper as [26]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The diference between the training dataset and the deployment data is called distribution shift, which often results in a performance drop [24,46,61].”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [61]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current LLMs are famously sensitive to small changes in prompt structure or they rely on shallow syntactic heuristics (McCoy, Pavlick, et al., 2019)”
From Machine Psychology (2023) · cites this paper as McCoy, Pavlick, et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A recurring concern with benchmarking is that models may use heuristics or shallow strategies to perform a task correctly without necessarily using the ability that is being tested [93,94].”
From Re-evaluating Theory of Mind evaluation in large language models (2025) · cites this paper as [93]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · cited 3×
“For the reasons we explain below, these heuristics are likely to be adopted by a statistical learner trained to perform NLI using standard training datasets such as SNLI (Bowman et al., 2015) or MNLI (Williams et al., 2018b).”
BackgroundMethodA Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
Accurate unlexicalized parsing
Klein, D., & Manning, C. D. (2003) · Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - ACL '03 · doi:10.3115/1075096.1075150 · Cited by 3,008
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049 · cited 1×
“In a similar vein, McCoy et al. (2019) found little evidence of compositional structure in the InferSent model (Conneau et al., 2017), which was trained on SNLI, even though other RNNs did learn clear compositional structure when trained on tasks that required such structure.”
MethodFrazier, L., & Rayner, K. (1982) · Cognitive Psychology · doi:10.1016/0010-0285(82)90008-1 · Cited by 1,767
The PASCAL Recognising Textual Entailment Challenge
Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture Notes in Computer Science · doi:10.1007/11736790_9 · Cited by 1,628
A Decomposable Attention Model for Natural Language Inference
Parikh, A., Täckström, O., Das, D., & Uszkoreit, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1244 · Cited by 1,296
The cognitive basis for linguistic structures1
Bever, T. G. (2013) · Language Down the Garden Path · doi:10.1093/acprof:oso/9780199677139.003.0001 · Cited by 1,288
AllenNLP: A Deep Semantic Natural Language Processing Platform
Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M., Schmitz, M., & Zettlemoyer, L. (2018) · Proceedings of Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/w18-2501 · Cited by 1,252
Enhanced LSTM for Natural Language Inference
Chen, Q., Zhu, X., Ling, Z.-H., Wei, S., Jiang, H., & Inkpen, D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1152 · Cited by 1,113
Annotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
Linzen, T., Dupoux, E., & Goldberg, Y. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00115 · Cited by 737
Self-citationThematic Roles Assigned along the Garden Path Linger
Christianson, K., Hollingworth, A., Halliwell, J. F., & Ferreira, F. (2001) · Cognitive Psychology · doi:10.1006/cogp.2001.0752 · Cited by 615
Hypothesis Only Baselines in Natural Language Inference
Poliak, A., Naradowsky, J., Haldar, A., Rudinger, R., & Van Durme, B. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2023 · Cited by 480
Colorless Green Recurrent Networks Dream Hierarchically
Gulordava, K., Bojanowski, P., Grave, E., Linzen, T., & Baroni, M. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1108 · Cited by 465
Self-citationAssessing BERT's Syntactic Abilities
Goldberg, Y. (2019) · arXiv · doi:10.48550/arxiv.1901.05287 · Cited by 385 · cited 1×
“First, the same pretrained BERT model we used (namely, the bert-base-uncased model) was found by Goldberg (2019) to achieve strong results on syntactic evaluations, suggesting that this pretrained model does have access to relevant syntactic information.”
MethodTargeted Syntactic Evaluation of Language Models
Marvin, R., & Linzen, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1151 · Cited by 334 · cited 1×
“…agreement, a phenomenon that requires a distinction to be made between the subject and direct object of a sentence (Linzen et al., 2016; Gulordava et al., 2018; Marvin and Linzen, 2018); it is unlikely that our fine-tuning step on MNLI substantially changed the model’s representational capabilities.”
BackgroundSelf-citationBreaking NLI Systems with Sentences that Require Simple Lexical Inferences
Glockner, M., Shwartz, V., & Goldberg, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2103 · Cited by 319 · cited 1×
“Some of these works focus on specific semantic phenomena, such as lexical inferences (Glockner et al., 2018) and quantifiers (Geiger et al., 2018), or biases based on specific words (Sanchez et al., 2018).”
BackgroundStress Test Evaluation for Natural Language InferencePreprint
Naik, A., Ravichander, A., Sadeh, N., Rosé, C. P., & Neubig, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00692 · Cited by 319
Analyzing the Behavior of Visual Question Answering Models
Agrawal, A., Batra, D., & Parikh, D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1203 · Cited by 297 · cited 2×
“Similar heuristics arise in visual question answering systems (Agrawal et al., 2016).”
BackgroundFine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., & Goldberg, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04207 · Cited by 283
What you can cram into a single vector: Probing sentence embeddings for linguistic properties
Conneau, A., Kruszewski, G., Lample, G., Barrault, L., & Baroni, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01070 · Cited by 263
Effects of merely local syntactic coherence on sentence processing
Tabor, W., Galantucci, B., & Richardson, D. (2004) · Journal of Memory and Language · doi:10.1016/j.jml.2004.01.001 · Cited by 261
A Fast Unified Model for Parsing and Sentence Understanding
Bowman, S. R., Gauthier, J., Rastogi, A., Gupta, R., Manning, C. D., & Potts, C. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1139 · Cited by 226
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
Adi, Y. (2024) · arXiv (Cornell University) · doi:10.57702/tcr06vve · Cited by 212
Manning, C. D., & MacCartney, B. (2009) · Cited by 209
Evaluating Compositionality in Sentence EmbeddingsPreprint
Dasgupta, I., Guo, D., Stuhlmüller, A., Gershman, S. J., & Goodman, N. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04302 · Cited by 145
Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation
Poliak, A., Haldar, A., Rudinger, R., Hu, J. E., Pavlick, E., White, A. S., & Van Durme, B. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1007 · Cited by 102
Self-citationInference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework
White, A. S., Rastogi, P., Duh, K., & Durme, B. V. (2017) · International Joint Conference on Natural Language Processing · Cited by 100
Entailment, intensionality and text understanding
Condoravdi, C., Crouch, D., de Paiva, V., Stolle, R., & Bobrow, D. G. (2003) · Proceedings of the HLT-NAACL 2003 workshop on Text meaning - · doi:10.3115/1119239.1119245 · Cited by 99
Do latent tree learning models identify meaningful structure in sentences?
Williams, A., Drozdov*, A., & Bowman, S. R. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00019 · Cited by 92
Rudinger, R., White, A. S., & Durme, B. V. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1067 · Cited by 88
Assessing Composition in Sentence Vector RepresentationsPreprint
Ettinger, A., Elgohary, A., Phillips, C., & Resnik, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03992 · Cited by 86
Analyzing Compositionality-Sensitivity of NLI Models
Nie, Y., Wang, Y., & Bansal, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016867 · Cited by 76
McCoy, R. T., Frank, R., & Linzen, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09091 · Cited by 58
Self-citationBehavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness
Sanchez, I., Mitchell, J., & Riedel, S. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1179 · Cited by 55
Lexicosyntactic Inference in Neural Models
White, A. S., Rudinger, R., Rawlins, K., & Durme, B. V. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1501 · Cited by 50
Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark
Nangia, N., & Bowman, S. R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1449 · Cited by 48
RNNs Implicitly Implement Tensor Product Representations
McCoy, R. T., Linzen, T., Dunbar, E., & Smolensky, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.08718 · Cited by 45
Self-citationCollecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation
Poliak, A., Haldar, A., Rudinger, R., Hu, J. E., Pavlick, E., White, A. S., & Van Durme, B. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5441 · Cited by 42
Self-citationVisual concepts and compositional voting
Wang, J., Zhang, Z., Xie, C., Zhou, Y., Premachandran, V., Zhu, J., Xie, L., & Yuille, A. (2018) · Annals of Mathematical Sciences and Applications · doi:10.4310/amsa.2018.v3.n1.a5 · Cited by 33
Syntactic/Semantic Structures for Textual Entailment Recognition
Mehdad, Y., Moschitti, A., & Zanzotto, F. M. (2010) · Cineca Institutional Research Information System (Tor Vergata University) · Cited by 32
The Fine Line between Linguistic Generalization and Failure in Seq2Seq-Attention Models
Weber, N., Shekhar, L., & Balasubramanian, N. (2018) · doi:10.18653/v1/w18-1004 · Cited by 30
Stress-Testing Neural Models of Natural Language Inference with Multiply-Quantified SentencesPreprint
Geiger, A., Cases, I., Karttunen, L., & Potts, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.13033 · Cited by 29
Non-Entailed Subsequences as a Challenge for Natural Language InferencePreprint
McCoy, R. T., & Linzen, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.12112 · Cited by 18
Self-citationVisual Concepts and Compositional VotingPreprint
Wang, J., Zhang, Z., Xie, C., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04451 · Cited by 16
Tense Manages to Predict Implicative Behavior in Verbs
Pavlick, E., & Callison-Burch, C. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1240 · Cited by 11
Self-citationCambridge: Parser Evaluation Using Textual Entailment by Grammatical Relation Comparison
Rimell, L., & Clark, S. R. L. (2010) · Cited by 9
Teaching Syntax by Adversarial Distraction
Kim, J., Malon, C. D., & Kadav, A. (2018) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/w18-5512 · Cited by 8
Analyzing Compositionality-Sensitivity of NLI ModelsPreprint
Nie, Y., Wang, Y., & Bansal, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07033 · Cited by 8
Research works printed but not linked: 2
- [22]22. Dan Klein and Christopher D. Manning. 2003. Ac- curate unlexicalized parsing. In Proceedings of the 41st Annual Meeting of the Association for Compu- tational Linguistics.
- [44]44. Aaron Steven White and Kyle Rawlins. 2018. The role of veridicality and factivity in clause selection. In Proceedings of the 48th Annual Meeting of the North East Linguistic Society.
Cited by
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
“The main disadvantages are the need for a new large dataset for every task, the potential for poor generalization out-of-distribution [MPL19], and the potential to exploit spurious features of the training data [GSL + 18, NK19].”
BackgroundShortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224
A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-1300 · Cited by 2,985
Metadata
- Authors
- McCoy, R. T., Pavlick, E., & Linzen, T.
- Affiliations
- Brown University (US)
- Johns Hopkins University (US)
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 3428-3448
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2951286828
- Cited by
- 944
- References
- 50
- Citations by year
- 2026: 43 · 2025: 42 · 2024: 69 · 2023: 199 · 2022: 183 · 2021: 213 · 2020: 162 · 2019: 32
- FWCI
- 56.54
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1746891)
- Johns Hopkins University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- natural language inference, MNLI, BERT, logical consequence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.