The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
An Efficient, Probabilistically Sound Algorithm for Segmentation and Word Discovery
Open accessAbstract
This paper presents a model-based, unsupervised algorithm for recovering word boundaries in a natural-language text from which they have been deleted. The algorithm is derived from a probability model of the source that generated the text. The fundamental structure of the model is specified abstractly so that the detailed component models of phonology, word-order, and word frequency can be replaced in a modular fashion. The model yields a language-independent, prior probability distribution on all possible sequences of all possible words over a given alphabet, based on the assumption that the input was generated by concatenating words from a fixed but unknown lexicon. The model is unusual in that it treats the generation of a complete corpus, regardless of length, as a single event in the probability space. Accordingly, the algorithm does not estimate a probability distribution on words; instead, it attempts to calculate the prior probabilities of various word sequences that could underlie the observed text. Experiments on phonemic transcripts of spontaneous speech by parents to young children suggest that our algorithm is more effective than other proposed algorithms, at least when utterance boundaries are given and the text includes a substantial number of short utterances.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
Harris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
An Introduction to Kolmogorov Complexity and Its Applications
Li, M., & Vitányi, P. (2019) · Texts in Computer Science · doi:10.1007/978-3-030-11298-1 · Cited by 3,681
Statistical Methods for Speech Recognition
Sun, D. X., & Jelinek, F. (1999) · Journal of the American Statistical Association · doi:10.2307/2670189 · Cited by 1,946
Stochastic Complexity in Statistical Inquiry
Rissanen, J. (1998) · WORLD SCIENTIFIC eBooks · doi:10.1142/0822 · Cited by 1,539
Word Segmentation: The Role of Distributional Cues
Saffran, J. R., Newport, E. L., & Aslin, R. N. (1996) · Journal of Memory and Language · doi:10.1006/jmla.1996.0032 · Cited by 1,405 · cited 2×
“Saffran, Newport, and Aslin (1996) suggest that children may segment utterances at low points of the transitional probability between adjacent syllables—that is, when a syllable occurs that is surprising given its predecessor.”
An Information Measure for Classification
Wallace, C. S., & Boulton, D. M. (1968) · The Computer Journal · doi:10.1093/comjnl/11.2.185 · Cited by 933
The child language data exchange system
MacWhinney, B., & Snow, C. (1985) · Journal of Child Language · doi:10.1017/s0305000900006449 · Cited by 861
Witten, I. H., & Bell, T. C. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.87000 · Cited by 729
The Psycho-Biology of Language
Tinker, M. A., & Zipf, G. K. (1936) · The American Journal of Psychology · doi:10.2307/1415586 · Cited by 728
Joos, M. (1936) · Language · doi:10.1017/408930 · Cited by 726
Inferring decision trees using the minimum description lenght principle
Ross Quinlan, J., & Rivest, R. L. (1989) · Information and Computation · doi:10.1016/0890-5401(89)90010-2 · Cited by 652
STOCHASTIC COMPLEXITY IN STATISTICAL INQUIRY
Dawid, A. P. (1991) · Bulletin of the London Mathematical Society · doi:10.1112/blms/23.1.106 · Cited by 651
An Introduction to Kolmogorov Complexity and Its Applications
Li, M., & Vitányi, P. (1993) · Springer eBooks · doi:10.1007/978-1-4757-3860-5 · Cited by 638
Gasarch, W. (1997) · ACM SIGACT News · doi:10.1145/262301.1040343 · Cited by 583
Distributional regularity and phonotactic constraints are useful for segmentation
Brent, M. R., & Cartwright, T. A. (1996) · Cognition · doi:10.1016/s0010-0277(96)00719-6 · Cited by 514
Self-citationLearning to Segment Speech Using Multiple Cues: A Connectionist Model
Christiansen, M. H., Allen, J., & Seidenberg, M. S. (1998) · Language and Cognitive Processes · doi:10.1080/016909698386528 · Cited by 356
Some Effects of Intermittent Silence
Miller, G. A. (1957) · The American Journal of Psychology · doi:10.2307/1419346 · Cited by 330
Church, K. W., & Gale, W. A. (1991) · Computer Speech & Language · doi:10.1016/0885-2308(91)90016-j · Cited by 234
Cartwright, T. A., & Brent, M. R. (1997) · Cognition · doi:10.1016/s0010-0277(96)00793-7 · Cited by 196
Self-citationStatistical methods for speech recognition
Jelinek, F. (1997) · Cited by 182
Bayesian learning of probabilistic language models
Stolcke, A. (1994) · Cited by 171
Redundancy Reduction as a Strategy for Unsupervised Learning
Redlich, A. (1993) · Neural Computation · doi:10.1162/neco.1993.5.2.289 · Cited by 142
Language acquisition, data compression and generalization
Wolff, J. G. (1982) · Language & Communication · doi:10.1016/0271-5309(82)90035-0 · Cited by 122
Models of Word Segmentation in Fluent Maternal Speech to Infants
Aslin, R. Ν., Woodward, J. Z., & LaMendola, N. P. (2014) · doi:10.4324/9781315806822-16 · Cited by 89
Dahan, D., & Brent, M. R. (1999) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.128.2.165 · Cited by 87
Self-citationAdvances in the computational study of language acquisition
Brent, M. R. (1996) · Cognition · doi:10.1016/s0010-0277(96)00779-2 · Cited by 76
Self-citationChild language data exchange system
(1984) · Journal of Child Language · doi:10.1017/s0305000900006139 · Cited by 61
The Unsupervised Acquisition of a Lexicon from Continuous SpeechPreprint
de Marcken, C. (1995) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9512002 · Cited by 58
Toward a Unified Model of Lexical Acquisition and Lexical Access
Brent, M. (1997) · Journal of Psycholinguistic Research · doi:10.1023/a:1025032825951 · Cited by 43
Self-citationThe Phonology of Parent-Child Speech
Ratner, N. B. (2021) · Psychology Press eBooks · doi:10.4324/9781315792668-7 · Cited by 22
Cited by
Event perception: a mind-brain perspective
Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.2.273 · Cited by 1,148
Event structure in perception and conception.
Zacks, J. M., & Tversky, B. (2001) · Psychological Bulletin · doi:10.1037//0033-2909.127.1.3 · Cited by 1,198
The role of exposure to isolated words in early vocabulary development
Brent, M. R., & Siskind, J. M. (2001) · Cognition · doi:10.1016/s0010-0277(01)00122-6 · Cited by 542
Self-citationA Bayesian framework for word segmentation: Exploring the effects of context
Goldwater, S., Griffiths, T. L., & Johnson, M. (2009) · Cognition · doi:10.1016/j.cognition.2009.03.008 · Cited by 431
Mommy and Me: Familiar Names Help Launch Babies Into Speech-Stream Segmentation
Bortfeld, H., Morgan, J. L., Golinkoff, R. M., & Rathbun, K. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.01531.x · Cited by 427
Metadata
- Authors
- Brent, M. R.
- Venue
- Machine Learning
- Published
- 1999-02
- Volume
- 34
- Issue
- 1-3
- Pages
- 71-105
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W2074546930
- Cited by
- 351
- References
- 31
- FWCI
- 9.36
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.