Distributional regularity and phonotactic constraints are useful for segmentation

“Word-based Strategies Olivier (1968) proposed a procedure that maintains a lexicon and uses it for segmenting.”

“At one extreme, the phonological component might segment perfectly, as assumed in most attempts to explain the syntactic and semantic components of lexical acquisition (e.g., Pinker, 1984; Siskind, this issue).”

“It has been proposed that such regularities play a role in early word discovery (Cutler, 1994, 1996; Cutler, Mehler, Norris, & Segui, 1994).”

“Jusczyk and Aslin (1995) familiarized 7.5-month-old American infants with two diierent monosyllabic words and then presented them with passages that either did or did not contain the familiar words in sentences.”

“Jusczyk, Cutler, and Redanz (1993) have found that 9-month-old American infants listen longer to words with strong-weak stress (the predominant pattern in English) than to words with weak-strong stress, but 6-month-olds show no preference.”

“Orthographic transcripts made by Bernstein-Ratner (1987) were taken from the CHILDES collection (MacWhinney & Snow, 1985) and transcribed phonetically.”

“For example, initial syllables of English content words rarely have reduced vowels (Cutler & Carter, 1987, estimate 10%).”

“In several experiments, 9-month-olds listened longer to words that are phonotactically permissible in their language than to those that are not, while 6-month-olds showed no preference (Friederici & Wessels, 1993; Jusczyk, Friederici, Wessels, Svenkerud, & Jusczyk, 1993).”

“Children learning other languages would use rhythmic patterns appropriate to their languages (Cutler et al., 1994).”

“MRL has been applied to language (e.g., Ellison, in press; Stolcke & Omohundro, 1994) and to modeling language acquisition (Brent, Gafos, & Cartwright, 1994).”

“Word-based Strategies Olivier (1968) proposed a procedure that maintains a lexicon and uses it for segmenting.”

“Further, French infants can distinguish phonemically identical sound sequences that diier only in whether they were spliced out of sentences in which they spanned a word boundary (Christophe et al., 1994).”

“It has been proposed that such regularities play a role in early word discovery (Cutler, 1994, 1996; Cutler, Mehler, Norris, & Segui, 1994).”

“For example, stop consonants tend to be aspirated more strongly when they occur word-initially than when they occur word-internally (Church, 1987).”

“…is statistical information derived from the distribution of patterns of sounds (Hayes and Clark, 1970; Goodsitt et al., 1993; Christophe et al., 1994; Brent and Cartwright, 1996; Saffran et al., 1996a; Saffran et al., 1996b; Saffran et al., 1997; Aslin et al., 1998; Christiansen et al., 1998).”

“VOL. 9, NO. 4, JULY 1998 Computation of Conditional Probabilities by Infants rely on a single type of information (see Brent & Cartwright, 1996; Christiansen, Allen, & Seidenberg, in press), such as frequency of co-occurrence or transitional probability between adjacent syllables.”

“One proposed solution to the word segmentation problem is that infants may be able to exploit statistical cues to word boundaries (Aslin, Woodward, LaMendola, & Bever, 1996; Brent & Cartwright, 1996; Hayes & Clark, 1970; Saffran, Newport, & Aslin, 1996).”

“As a result, each time a word appears in the input, the model receives a consistent sequence of tokenized input (e.g., Harrington et al., 1989; Aslin, Woodward, LaMendola & Bever, 1996; Brent & Cartwright, 1996; Brent, 1999).”

“…interacting schemata based on prosodic and phonetic pattern-matching are at work in the infant to accomplish utterance and word segmentation (Brent and Cartwright, 1996; Fisher and Tokura, 1996; Morgan, Meier, and Newport, 1989; Saffran, Newport, and Aslin, 1996; Seidenberg and McClelland,…”

“Such mechanisms have been proposed (Peters, 1983; Pinker, 1984) and have been formalized and shown to be effective via computer simulation (Brent, 1999; Brent & Cartwright, 1996).”

“The first, derived from signal detection theory, is becoming standard in this area (Brent & Cartwright, 1997; Christiansen, Allen & Seidenberg, in press).”

“Among the potential cues to word boundaries are (1) allophonic variations (Hohne & Jusczyk, 1994; Jusczyk, Hohne, & Bauman, in press-a), (2) distributional regularities (Brent & Cartwright, 1996; Goodsitt, Morgan, & Kuhl, 1993; Saffran, Aslin, & Newport, 1996), (3) prosody (Echols, Crowhurst, & Childers, 1997; Jusczyk…”

“…algorithm is intended to provide a more realistic simulation of human word segmentation than earlier batch learning algorithms (de Marcken, 1995; Brent and Cartwright, 1996), which assume that the entire corpus of data is available to the learner at once (i.e., the learner may iterate over the…”

“A similar statistical approach has been proposed by Brent and colleagues in word and morpheme discovery (Brent, 1993; Brent & Cartwright, 1996), though these implementations require a perhaps implausibly large memory for unanalysed sequences.”

“analysis [30], segmentation [ 31 ,24] and grammar induction [32,33]”

“Modeling provides a way of examining and combining probabilistic cues in order to determine if this information can be used to segment words out of the speech stream (Brent & Cartwright, 1996; Christiansen, Allen, & Seidenberg, 1998).”

“However, none of these cues appear to be sufficient (singly, or in combination) to identify all of the critical boundaries in connected speech (Brent and Cartwright, 1996; Christiansen et al., 1998).”

“Recent psycholinguistic research has given evidence that these features may be used as cues for word boundaries during language acquisition (see Brent & Cartwright, 1996; Christiansen, Allen, & Seidenberg, in press; Jusczyk, 1997; McDonald, 1997).”

“Some theorists focus on applying probability to learning parameters of Chomskyan Universal Grammar [54,55]; others focus on learning relatively simple aspects of language, such as syntactic or semantic categories, or approximate morphological decomposition, with relatively weak prior assumptions [44, 56 ,57].”

“However, as Brent and Cartwright (1996) pointed out, semantic categorization is a substantial problem in its own right and does not seem a robust foundation for phonological development.”

“The hypothesis tested in Experiment 2 is that a biphone with low within-word probability and high between-word probability is an efficient word onset cue ( Brent & Cartwright, 1996; Cairns, Shillcock, Chater, & Levy, 1997; Gaygen & Luce, submitted; Mattys et al., 1999).”

“…of speech and language can provide a strong foundation for binding input into perceptually coherent and meaningful units like words (Figure 4C; Brent and Cartwright, 1996; McQueen, 1998; Saffran et al., 1996), which may not be as readily identifiable using amplitude-based temporal landmarks…”

“While we and a number of other investigators (e.g. Brent & Cartwright, 1996;”

“Brent and Cartwright (1996) conducted a series of computational studies of vocabulary acquisition, in which they used a Minimum Representation Length (Rissanen, 1989) tech­ nique to learn a vocabulary from selections of the CHILDES database (MacWhinney & Snow, 1985), transcribed in such a way that all word boundary”

“For example, Brent and Cartwright (1997) show how morphological structure can be found within isolated words, Wolff (1977; see also Redlich, 1993) considers how higher level structure can be found automatically in text.”

“Computer simulations have shown that this strategy works well on a variety of real-life corpora, using a variety of algorithms (Batchelder, 2002; Brent, 1999; Brent & Cartwright , 1996; Cairns, Shillcock, Chater, & Levy, 1997; Christiansen, Allen, & Seidenberg, 1998; Venkataraman, 2001).”

“Considerable work has focused on how and when infants find the boundaries between individual words in speech (e.g., Brent & Cartwright, 1996; Johnson & Jusczyk, 2001; Jusczyk & Aslin, 1995; Jusczyk, Houston, & Newsome, 1999; Myers et al., 1996; Saffran, Aslin, & Newport, 1996).”

“…acoustic cues to word onsets (Lehiste, 1960; Nakatani & Dukes, 1977) or from knowledge of statistical regu- larities of lexical items (such as distributional regularity, phonotactics, or metrical stress; see Brent & Cartwright, 1996; Cairns, Shillcock, Chater, & Levy, 1997; Cutler & Norris, 1988).”

“…models of corpora and speakers, it has been found that “distributional regularity and phonotactic constraints are useful for seg-mentation” (Brent and Cartwright, 1996). de Mar-cken (1996) proposed to deconstruct text recursively from sentences over words and morphs into characters through…”

“…statistical sponges comes largely from studies focusing on, e.g. the discovery of word boundaries (Saffran, Newport, & Aslin, 1996), phonotactic constraints (Brent & Cartwright, 1996), and the assignment of words to grammatical categories (Mintz, Newport, & Bever, 1995; cf. Brill & Marcus, 1992).”

“How do infants learn to segment fluent speech into a series of words? Statistical strategies were proposed for the segmentation of words, based on distributional properties over sub-lexical units like phonemes or syllables (e.g., Harris, 1955; Brent & Cartwright, 1996; Dahan & Brent, 1999; Batchelder, 2002; Swingley,…”

“…word-like units on the basis of the transitional probabilities between consecutive syllables (Saffran, Aslin, & Newport, 1996; though see Perruchet & Vinter, 1998, for an alternative interpretation of these results, and Brent & Cartwright, 1996; Dahan & Brent, 1999, for an alternative model).”

“Brent and Cartwright (1996) suggest leaving such words out of a (training) corpus because they occur in isolation more often than ordinary words, their use is highly idiosyncratic, and there is no standardised orthographic spelling for many of them.”

“Brent and colleagues (Brent & Cartwright, 1996; Cartwright & Brent, 1994) demonstrate the utility of supplying a model of segmentation with categorical phonotactic information consisting of all the legal initial and final consonant clusters found in the English lexicon.”

“…researchers have countered that the linguistic input to children does contain sufficient information to account for their linguistic representations (Brent & Cartwright, 1996; Seidenberg, 1997), and moreover, that this information can be extracted by general purpose learning mechanisms (Elman,…”

“The recurring sound sequences constituting words can be distinguished, on statistical grounds, from the more accidental sound sequences across word boundaries (Brent and Cartwright, 1996; Harris, 1955; Redlich, 1993).”

“We will begin with the DR model of Brent and Cartwright (1996) .”

“We used the MDL framework because it has been successfully applied to other language learning problems (e.g., Ellison, in press; Stolcke and Omohundro, 1994) and to modeling other aspects of language acquisition (Brent and Cartwright, 1996).”

“…grammar performs unsupervised word segmentation with an 87% token f-score on the standard Brent version of the Bernstein-Ratner corpus (Bernstein-Ratner, 1987; Brent and Cartwright, 1996), which is an error reduction of over 35% compared to the best previously reported results on this corpus.”

“Recently, a number of computational learning models have pursued probabilistic approaches to language learning (e.g. Brent & Cartwright, 1996; Eisner, 2002; Niyogi & Berwick, 1996).”

“…a conservative measure of word segmentation in which a word is considered to be correctly segmented only if a lexical boundary is predicted at the beginning and at the end of that word without any boundaries being predicted word-internally (Brent & Cartwright, 1996; Christiansen et al. , 1998).”

“Several behavioral experiments have reported evidence that infants and adults readily learn statistically defined patterns in auditory input sequences (Pena, Bonatti, Nespor, & Mehler, 2002; Seidenberg, MacDonald, & Saffran, 2002; Brent, 1999; Saffran, Johnson, Aslin, & Newport, 1999; Aslin, Saffran, & Newport, 1998;…”

“Thus, as noted above, phonotactic cues (Brent & Cartwright, 1996; Cairns, Shillcock, Chater, & Levy, 1997) and prosodic stress cues (Cutler, 1990; 1994; Cutler & Norris, 1988) have been promoted as useful for segmenting English words.”

“…First, it demonstrates the applicability and competitiveness of a conservative traditional approach for a task for which nontraditional approaches have been proposed even recently (Brent, 1999; Brent and Cartwright, 1996; de Marcken, 1995; Elman, 1990; Christiansen, Allen, and Seidenberg, 1998).”

“…and its relation to the phonological structure of the native language of the participants, provides a potentially rich source of information that learners may draw upon in performing experimental tasks (Brent & Cartwright, 1996; Friederici & Wessels, 1993; Mattys, J uscz yk, Luce, & Morgan, 1999).”

“As formulated by Brent and Cartwright (1996), “distributional regularity refers to the intuition that sound sequences that occur frequently and in a variety of contexts are better candidates for the lexicon than those that occur rarely and in few contexts”.”

“Brent and coworkers have produced work on segmentation (Brent & Cartwright, 1997), acquisition of syntactic categories (Brent, 1997), unsupervised learning of morphology (Snover & Brent, 2001) and learning of lexical syntax (Brent, 1991, 1993).”

“Peter Ford Dominey”

“Recent experimental and computational research has demonstrated the value of statistical and distributional information , in both the development of language abilities, and the adult system (Brent & Cartwright, 1996; Cairns, Shillcock, Chater & Levy, 1997; Saffran, Aslin & Newport, 1996).”

“…2005; Johnson & Jusczyk, 2001), phrase-level prosody (Shukla, Nespor & Mehler, 2007), sentence position (Seidl & Johnson, 2006), language-specific phonotactic constraints (Brent & Cartwright, 1996), and concurrently presented visual information (Cunillera, Cµmara, Laine & Rodríguez-Fornells, 2010).”

“As in Brent and Cartwright ( 1996 ) model , unit extraction depends on the overall consistency of the segmentation , because the units , which stands for attentional chunks , are discrete and disjunctive .”

“Our approach differs from current approaches in machine learning (see, e.g., Brent and Cartwright, 1996; Brent, 1999; Johnson, 2008; Goldwater et al., 2009; Robinet et al., 2011; Synnaeve et al., 2014) in two important ways.”

“Much research in unsupervised language learning is concerned with bootstrapping syntactic structure on the basis of pattern similarity and statistics from large language corpora (e.g., Finch & Chater, 1994; Brent & Cartwright, 1996; van Zaanen, 2000).”

“…practical generalization of the universal law [of generalization]: if we randomly pick items a and b , where we allow the most complex objects, then with overwhelming probability the universal law of generalization holds with the internal psychological space metric being the information metric.”

“As to computational models of word segmentation, several techniques have been implemented, including minimum description length compression (Brent & Cartwright, 1996), mutual information over syllables (Swingley, 2005), and syllable chunking (Perruchet & Vinter, 1998; Batchelder, 2002).”

“It is unlikely that listeners rely on only one kind of cue when segmenting speech (e.g., Cutler et al., 1986; Grosjean and Gee, 1987; Brent and Cartwright, 1996; Vroomen et al., 1996; McQueen, 1998; Sanders and Neville, 2000).”

“For example, consider spoken language, wherein distinct syllables transition from one to another in a continuous stream without pauses or demarcations between words (30).”

“…the performance of the model we used the complementary measures of PRECISION and RECALL, which have been used as conservative measures of model performance in previous research (e.g. Batchelder, 2002; Brent & Cartwright, 1996 ; Christiansen et al., 1998 ; Hockema, 2006 ; Venkataraman, 2001).”

“Cartwright and Brent Cartwright .and Brent, 1994; Brent and Cartwright, 1996 , for instance, propose to segment the text data by following a MDL approach.”

“Cairns et al. (1997) did not regard the diphone model as a suitable model for acquisition, because calculating the model parameters depends on knowing the relative frequency with which word boundaries span different diphones.”

“8Some models that take on the task of word segmentation include Allen and Christiansen (1996), Brent and Cartwright (1996) Cairns, Shillcock, Chater, and Levy (1997), and Brent (1999).”