The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Unsupervised learning of natural languages
Open accessAbstract
We address the problem, fundamental to linguistics, bioinformatics, and certain other disciplines, of using corpora of raw symbolic sequential data to infer underlying rules that govern their production. Given a corpus of strings (such as text, transcribed speech, chromosome or protein sequence data, sheet music, etc.), our unsupervised algorithm recursively distills from it hierarchically structured patterns. The adios(automatic distillation of structure) algorithm relies on a statistical method for pattern extraction and on structured generalization, two processes that have been implicated in language acquisition. It has been evaluated on artificial context-free grammars with thousands of rules, on natural languages as diverse as English and Chinese, and on protein data correlating sequence with function. This unsupervised algorithm is capable of learning complex syntax, generating grammatical novel sentences, and proving useful in other fields that call for structure discovery from raw data, such as bioinformatics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Introduction to automata theory, languages and computation
(1981) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(81)90068-9 · Cited by 10,699
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
(1995) · Cited by 6,864
Statistical learning by 8-month-old infants
Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996) · Science · doi:10.1126/science.274.5294.1926 · Cited by 5,659
A universal algorithm for sequential data compression
Ziv, J., & Lempel, A. (1977) · IEEE Transactions on Information Theory · doi:10.1109/tit.1977.1055714 · Cited by 5,456
Harris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Knowledge of Language: Its Nature, Origin, and Use.
Hornstein, N., & Chomsky, N. (1988) · The Philosophical Review · doi:10.2307/2185417 · Cited by 4,650
Foundations of Language: Brain, Meaning, Grammar, Evolution
Jackendoff, R. (2002) · doi:10.1093/acprof:oso/9780198270126.001.0001 · Cited by 3,244
Formulaic Language and the Lexicon
Wray, A. (2002) · doi:10.1017/cbo9780511519772 · Cited by 2,873
Baker, C. F., Fillmore, C. J., & Lowe, J. B. (1998) · Proceedings of the 36th annual meeting on Association for Computational Linguistics - · doi:10.3115/980845.980860 · Cited by 2,518
Rethinking Innateness: A Connectionist Perspective on Development
Harris, M. J., Karmiloff‐Smith, A., Parisi, D., & Plunkett, K. (1996) · Medical Entomology and Zoology · Cited by 1,990
Constructions: A new theoretical approach to language
Goldberg, A. E. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00080-9 · Cited by 1,812
守屋, 悦. (1980) · 情報処理 · Cited by 1,791
Rethinking Innateness: A Connectionist Perspective on Development
Elman, J., Bates, E., Johnson, M. H., Karmiloff-Smith, A., Parisi, D., & Plunkett, K. (1996) · doi:10.7551/mitpress/5929.001.0001 · Cited by 1,511
Chomsky, N. (1975) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 1,488
The idiom principle and the open choice principle
ERMAN, B., & WARREN, B. (2000) · Text - Interdisciplinary Journal for the Study of Discourse · doi:10.1515/text.1.2000.20.1.29 · Cited by 1,349
Natural language and natural selection
Pinker, S., & Bloom, P. (1990) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00081061 · Cited by 1,330
Rule Learning by Seven-Month-Old Infants
Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1999) · Science · doi:10.1126/science.283.5398.77 · Cited by 1,244
Grammatical Constructions and Linguistic Generalizations: The What's X Doing Y? Construction
Kay, P., & Fillmore, C. J. (1999) · Language · doi:10.2307/417472 · Cited by 1,094
The functions of formulaic language: an integrated model
Wray, A., & Perkins, M. R. (2000) · Language & Communication · doi:10.1016/s0271-5309(99)00015-4 · Cited by 866
Variability and Detection of Invariant Structure
Gómez, R. L. (2002) · Psychological Science · doi:10.1111/1467-9280.00476 · Cited by 840
Psychology and language : an introduction to psycholinguistics
Clark, H. H., & Clark, E. V. (1977) · Cited by 828
Methods in Structural Linguistics
McQuown, N. A., & Harris, Z. S. (1952) · Language · doi:10.2307/409684 · Cited by 804
The ATIS spoken language systems pilot corpus
Hemphill, C. T., Godfrey, J. J., & Doddington, G. R. (1990) · Proceedings of the workshop on Speech and Natural Language - HLT '90 · doi:10.3115/116580.116613 · Cited by 790
Foundations of Cognitive Grammar: Volume I: Theoretical Prerequisites
Langacker, R. W. (1987) · Cited by 755
Magnitude Estimation of Linguistic Acceptability
Bard, E. G., Robertson, D., & Sorace, A. (1996) · Language · doi:10.2307/416793 · Cited by 721
Joshi, A. K., & Schabes, Y. (1997) · Handbook of Formal Languages · doi:10.1007/978-3-642-59126-6_2 · Cited by 715
Sensitivity to grammatical structure in so-called agrammatic aphasics
Linebarger, M. C., Schwartz, M. F., & Saffran, E. M. (1983) · Cognition · doi:10.1016/0010-0277(83)90015-x · Cited by 697
Pattern Grammar: A corpus-driven approach to the lexical grammar of English
Hunston, S., & Francis, G. (2000) · Cited by 696
Applications of stochastic context-free grammars using the Inside-Outside algorithm
Lari, K., & Young, S. J. (1991) · Computer Speech & Language · doi:10.1016/0885-2308(91)90009-f · Cited by 639
Redundancy reduction revisited
Barlow, H. (2001) · Network: Computation in Neural Systems · doi:10.1080/net.12.3.241.253 · Cited by 620
A construction based analysis of child directed speech
Cameron‐Faulkner, T., Lieven, E., & Tomasello, M. (2003) · Cognitive Science · doi:10.1207/s15516709cog2706_2 · Cited by 592
Cai, C. Z. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg600 · Cited by 576
Infant artificial language learning and language acquisition
Gómez, R. L., & Gerken, L. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01467-4 · Cited by 506
The Empirical Base of Linguistics: Grammaticality Judgments and Linguistic Methodology
Schütze, C. T. (2016) · Cited by 469
Evolution of Universal Grammar
Nowak, M. A., Komarova, N. L., & Niyogi, P. (2001) · Science · doi:10.1126/science.291.5501.114 · Cited by 453
Signal-Driven Computations in Speech Processing
Peña, M., Bonatti, L. L., Nespor, M., & Mehler, J. (2002) · Science · doi:10.1126/science.1072901 · Cited by 451
Corpus-based induction of syntactic structure: models of dependency and constituency
Klein, D., & Manning, C. D. (2004) · Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04 · doi:10.3115/1218955.1219016 · Cited by 440
The item-based nature of children’s early syntactic development
Tomasello, M. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01462-5 · Cited by 419
The estimation of stochastic context-free grammars using the Inside-Outside algorithm
Lari, K., & Young, S. J. (1990) · Computer Speech & Language · doi:10.1016/0885-2308(90)90022-x · Cited by 379
Proceedings of EUROSPEECH-1991.
Lickley, R. J., Shillcock, R. C., & Bard, E. G. (1991) · Cited by 374
Finite-State Language Processing
Covington, M. A., Roche, E., & Schabes, Y. (1999) · Language · doi:10.2307/417760 · Cited by 354
Language Acquisition and Use: Learning and Applying Probabilistic Constraints
Seidenberg, M. S. (1997) · Science · doi:10.1126/science.275.5306.1599 · Cited by 338
Probabilistic Top-Down Parsing and Language Modeling
Roark, B. (2001) · Computational Linguistics · doi:10.1162/089120101750300526 · Cited by 329
(1998) · MPG.PuRe (Max Planck Society) · Cited by 293
Immediate-head parsing for language models
Charniak, E. (2001) · Proceedings of the 39th Annual Meeting on Association for Computational Linguistics - ACL '01 · doi:10.3115/1073012.1073029 · Cited by 293
Inside-outside reestimation from partially bracketed corpora
Pereira, F., & Schabes, Y. (1992) · Proceedings of the 30th annual meeting on Association for Computational Linguistics - · doi:10.3115/981967.981984 · Cited by 284
Inducing probabilistic grammars by Bayesian model merging
Stolcke, A., & Omohundro, S. (1994) · Lecture Notes in Computer Science · doi:10.1007/3-540-58473-0_141 · Cited by 230
Syntactic Nuts: Hard Cases, Syntactic Theory, and Language Acquisition
Culicover, P. W. (1999) · Medical Entomology and Zoology · Cited by 210
Unsupervised Language AcquisitionPreprint
de Marcken, C. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9611002 · Cited by 207
On the Emergence of Grammar From the Lexicon
(2013) · The Emergence of Language · doi:10.4324/9781410602367-7 · Cited by 191
Natural language grammatical inference with recurrent neural networks
Lawrence, S., Giles, C. L., & Fong, S. (2000) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.842255 · Cited by 168
Unitization during category learning.
Goldstone, R. L. (2000) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037//0096-1523.26.1.86 · Cited by 166
Exploiting Syntactic Structure for Language ModelingPreprint
Chelba, C. I., & Jelinek, F. (1998) · Cited by 163
Unsupervised induction of stochastic context-free grammars using distributional clustering
Clark, A. (2001) · doi:10.3115/1117822.1117831 · Cited by 149
van Zaanen, M. M. (2000) · International Conference on Computational Linguistics · doi:10.3115/992730.992785 · Cited by 147
Does Grammar Start Where Statistics Stop?
Seidenberg, M. S., MacDonald, M. C., & Saffran, J. R. (2002) · Science · doi:10.1126/science.1078094 · Cited by 142
Redundancy Reduction as a Strategy for Unsupervised Learning
Redlich, A. (1993) · Neural Computation · doi:10.1162/neco.1993.5.2.289 · Cited by 142
The Bible as a Parallel Corpus: Annotating the ‘Book of 2000 Tongues’
Resnik, P., Olsen, M. B., & Diab, M. (1999) · Computers and the Humanities · doi:10.1023/a:1001798929185 · Cited by 125
What is the computational goal of the neocortex
Barlow, H. B. (1994) · Cited by 121
Carrasco, R. C., & Oncina, J. (1994) · Lecture notes in computer science · doi:10.1007/3-540-58473-0 · Cited by 120
A Bit of Progress in Language Modeling Extended VersionPreprint
Goodman, J. (2001) · Cited by 113
Formal grammar and information theory: together again?
Pereira, F. M. B. (2000) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2000.0583 · Cited by 110
The Character of Natural Language Semantics
Pietroski, P. M. (2003) · Oxford University Press eBooks · doi:10.1093/oso/9780199250578.003.0007 · Cited by 100
Using a stochastic context-free grammar as a language model for speech recognition
Jurafsky, D., Wooters, C., Segal, J., Stolcke, A., Fosler, E., Tajchaman, G., & Morgan, N. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1995.479396 · Cited by 89
Unsupervised Language Acquisition: Theory and PracticePreprint
Clark, A. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0212024 · Cited by 81
A memory-based model of syntactic analysis: data-oriented parsing
Scha, R. J. H., Bod, R., & Sima’an, K. (1999) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/095281399146481 · Cited by 72
Constraining the neural representation of the visual world
Edelman, S. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01854-4 · Cited by 44
Self-citationUnsupervised Context Sensitive Language Acquisition from a Large Corpus
Solan, Z., Horn, D. G., Ruppin, E., & Edelman, S. (2003) · Cited by 29
Self-citationAutomatic Acquisition and Efficient Representation of Syntactic Structures
Solan, Z., Ruppin, E., Horn, D. G., & Edelman, S. (2002) · Cited by 29
Self-citation(2001)
Cited by
The myth of language universals: Language diversity and its importance for cognitive science
Evans, N., & Levinson, S. C. (2009) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999094x · Cited by 2,623
The Oxford Handbook of Linguistic Analysis
(2009) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199544004.001.0001 · Cited by 639
The Cartography of Syntactic Structures
Cinque, G., & Rizzi, L. (2012) · The Oxford Handbook of Linguistic Analysis · doi:10.1093/oxfordhb/9780199544004.013.0003 · Cited by 639
Social Feedback to Infants’ Babbling Facilitates Rapid Phonological Learning
Goldstein, M. H., & Schwade, J. A. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02117.x · Cited by 620
Representing word meaning and order information in a composite holographic lexicon.
Jones, M. N., & Mewhort, D. J. K. (2007) · Psychological Review · doi:10.1037/0033-295x.114.1.1 · Cited by 613
Metadata
- Authors
- Solan, Z., Horn, D., Ruppin, E., & Edelman, S.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2005-08-08
- Volume
- 102
- Issue
- 33
- Pages
- 11629-11634
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2143296986
- Cited by
- 346
- References
- 72
- FWCI
- 23.02
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.