Using Computational Models to Test Syntactic Learnability
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
We studied the learnability of English filler-gap dependencies and the “island” constraints on them by assessing the generalizations made by autoregressive (incremental) language models that use deep learning to predict the next word given preceding context. Using factorial tests inspired by experimental psycholinguistics, we found that models acquire not only the basic contingency between fillers and gaps, but also the unboundedness and hierarchical constraints implicated in the dependency. We evaluated a model’s acquisition of island constraints by demonstrating that its expectation for a filler-gap contingency is attenuated within an island environment. Our results provide empirical evidence against the argument from the poverty of the stimulus for this particular structure.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Random effects structure for confirmatory hypothesis testing: Keep it maximal
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013) · Journal of Memory and Language · doi:10.1016/j.jml.2012.11.001 · Cited by 10,328 · cited 2×
“To test for this interaction, we ran mixed-effects linear regression models, fit on raw surprisal values with sum-coded conditions and by-item random slopes (Barr et al. 2013), and looked for significantly negative interaction terms.”
Chomsky, N. (1957) · doi:10.1515/9783112316009 · Cited by 7,885
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Meaningful Differences in the Everyday Experience of Young American Children
(1996) · Choice Reviews Online · doi:10.5860/choice.33-3435 · Cited by 4,103
Language identification in the limit
Gold, E. M. (1967) · Information and Control · doi:10.1016/s0019-9958(67)91165-5 · Cited by 3,572 · cited 2×
“…1992, Culbertson, Smolensky, and Legendre 2012); crosslinguistic analyses of phenomena (Richards 2001); analytic arguments about learnability (Gold 1967, Chomsky 1979); corpus analyses (Pullum and Scholz 2002); and statistical and computational modeling (Perfors, Regier, and Tenenbaum 2006,…”
McClelland, J. L., Rumelhart, D. E., & AU (1987) · doi:10.7551/mitpress/5237.001.0001 · Cited by 3,315
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Cited by
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
A hierarchy of linguistic predictions during natural language comprehension
Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201968119 · Cited by 343
Large language models demonstrate the potential of statistical learning in language
Contreras Kallens, P., Kristensen‐McLachlan, R. D., & Christiansen, M. H. (2023) · Cognitive Science · doi:10.1111/cogs.13256 · Cited by 130
“Through careful analyses of their output, we can assess just how much can be learned from the statistical regularities of the linguistic environment (Futrell et al., 2019; Wilcox, Futrell, & Levy, 2022).”
BackgroundLanguage Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105
Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
Warstadt, A., Mueller, A., Choshen, L., Wilcox, E., Zhuang, C., Ciro, J., Mosquera, R., Paranjabe, B., Williams, A., Linzen, T., et al. (2023) · Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-babylm.1 · Cited by 74
Self-citation
Metadata
- Authors
- Wilcox, E. G., Futrell, R., & Levy, R.
- Venue
- Linguistic Inquiry
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 55
- Issue
- 4
- Pages
- 805-848
- ISSN
- 0024-3892 (ISSN-L)
- 1530-9150
- OpenAlex
- W4303579584
- Cited by
- 62
- References
- 22
- FWCI
- 51.79
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using Computational Models to Test Syntactic Learnability. https://app.cochat.ai/papers/10.1162/ling_a_00491
- The paper
- Wilcox, E. G., Futrell, R., & Levy, R. (2024). Using Computational Models to Test Syntactic Learnability. Linguistic Inquiry. https://doi.org/10.1162/ling_a_00491
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.