The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index, and 1 are not checked yet.
Syntactic Structure from Deep Learning
Open accessAbstract
Modern deep neural networks achieve impressive performance in engineering applications that require extensive linguistic skills, such as machine translation. This success has sparked interest in probing whether these models are inducing human-like grammatical knowledge from the raw data they are exposed to and, consequently, whether they can shed new light on long-standing debates concerning the innate structure necessary for language acquisition. In this article, we survey representative studies of the syntactic abilities of deep networks and discuss the broader implications that this work has for theoretical linguistics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“That said, careful analysis (27) indicates that the deep network’s sensitivity to grammatical structure is still imperfect and only partially understood.”
From Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models (2020) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Chomsky, N. (1957) · doi:10.1515/9783112316009 · Cited by 7,885
The faculty of language: what is it, who has it, and how did it evolve?
Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002) · Science · doi:10.1126/science.298.5598.1569 · Cited by 5,189
Meaningful Differences in the Everyday Experience of Young American Children
(1996) · Choice Reviews Online · doi:10.5860/choice.33-3435 · Cited by 4,103
Connectionism and cognitive architecture: A critical analysis
Fodor, J. A., & Pylyshyn, Z. W. (1988) · Cognition · doi:10.1016/0010-0277(88)90031-5 · Cited by 4,022
Distributed and overlapping representations of faces and objects in ventral temporal cortex
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001) · Science · doi:10.1126/science.1063736 · Cited by 3,940
Research works printed but not linked: 27
- Chaves RP., 2020, Proc. Soc. Comput. Linguist., 3, 20
- Chomsky N., 1965, Aspects of the Theory of Syntax
- Chomsky N., 1986, Knowledge of Language: Its Nature, Origin, and Use
- Chomsky N., 1995, The Minimalist Program
- Chomsky N, 1963, In Handbook of Mathematical Psychology, 2, 269
- Chowdhury S, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 133
- Churchland P., 1989, A Neurocomputational Perspective: The Nature of Mind and the Structure of Science
- Devlin J, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 4171
- Elman JL, 1998, Rethinking Innateness: A Connectionist Perspective on Development
- Goldberg A., 2019, Explain Me This: Creativity, Competition, and the Partial Productivity of Constructions
- Hewitt J, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, 4129
- Jurafsky D, 2008, Speech and Language Processing
- Lakretz Y, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 11
- Lasnik H, 2017, Oxford Handbook of Universal Grammar, 221
- Linzen T, 2018, The 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP: Proceedings of the First Workshop
- Linzen T, 2019, The BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP at ACL 2019: Proceedings of the Second Workshop
- Linzen T, 2018, Proceedings of the 40th Annual Conference of the Cognitive Science Society, 692
- McCoy T, 2018, Proceedings of the 40th Annual Conference of the Cognitive Science Society, 2093
- Mitchell TM., 1980, The need for biases in learning generalizations
- Radford A, 2019, Language models are unsupervised multitask learners
- Raffel C, Shazeer N, Roberts A, Lee K, Narang S, et al. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv:1910.10683 [cs.LG] · arXiv
- Ross J., 1967, Constraints on variables in syntax
- Socher R, 2011, ICML'11: Proceedings of the 28th International Conference on Machine Learning, 129
- Sutskever I, 2014, NIPS'14: Proceedings of the 27th International Conference on Neural Information Processing Systems, 2, 3104
- van Schijndel M, 2018, Proceedings of the 40th Annual Conference of the Cognitive Science Society, 2603
- Vaswani A, 2017, NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems, 6000
- Weston J., 2016, NIPS'16: Proceedings of the 30th International Conference on Neural Information Processing Systems, 829
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Shared computational principles for language processing in humans and deep language models
Goldstein, A., Zada, Z., Buchnik, E., Schain, M., Price, A., Aubrey, B., Nastase, S. A., Feder, A., Emanuel, D., Cohen, A., et al. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01026-4 · Cited by 447
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
Do large language models resemble humans in language use?Preprint
Cai, Z. G., Duan, X., Haslett, D. A., Wang, S., & Pickering, M. J. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/s49qv · Cited by 131
What Artificial Neural Networks Can Tell Us about Human Language Acquisition
Warstadt, A., & Bowman, S. R. (2022) · Algebraic Structures in Natural Language · doi:10.1201/9781003205388-2 · Cited by 111
Metadata
- Authors
- Linzen, T., & Baroni, M.
- Affiliations
- New York University (US)
- Institució Catalana de Recerca i Estudis Avançats (ES)
- ORCID
- Baroni, Marco
- Venue
- Annual Review of Linguistics
- Publisher
- Annual Reviews
- Published
- 2021-01-14
- Volume
- 7
- Issue
- 1
- Pages
- 195-212
- ISSN
- 2333-9683 (ISSN-L)
- 2333-9691
- OpenAlex
- W3018827121
- Cited by
- 202
- References
- 48
- Citations by year
- 2026: 17 · 2025: 31 · 2024: 33 · 2023: 39 · 2022: 37 · 2021: 36
- FWCI
- 50.70
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Annual Review of Linguistics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Psychology › Developmental and Educational Psychology › Language Development and Disorders
- Keywords
- syntactic structure, deep neural network, grammatical knowledge, language acquisition, innate language faculty, machine translation, linguistic theory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Syntactic Structure from Deep Learning. https://app.cochat.ai/papers/10.1146/annurev-linguistics-032020-051035
- The paper
- Linzen, T., & Baroni, M. (2021). Syntactic Structure from Deep Learning. Annual Review of Linguistics. https://doi.org/10.1146/annurev-linguistics-032020-051035
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.