The DIVA model: A neural theory of speech acquisition and production

“Similarly, comparisons of auditory responses during self-produced speech and while listening to recordings of one’s own speech indicate attenuation of auditory cortex responses during speech production (Curio, Neuloh, Numminen, Jousmaki, & Hari, 2000; Heinks-Maldonado, Mathalon, Gray, & Ford, 2005;…”

“1 Performance of these motor programs relies on contributions from bilateral premotor, motor, and subcortical regions (see below), areas that are active bilaterally during normal speech production.”

“Blood oxygenation level-dependent (BOLD) functional magnetic resonance imaging (fMRI; Belliveau et al., 1992; Kwong et al., 1992; Ogawa et al.,”

“Similarly, comparisons of auditory responses during self-produced speech and while listening to recordings of one’s own speech indicate attenuation of auditory cortex responses during speech production (Curio, Neuloh, Numminen, Jousmaki, & Hari, 2000; Heinks-Maldonado, Mathalon, Gray, & Ford, 2005;…”

“1 Performance of these motor programs relies on contributions from bilateral premotor, motor, and subcortical regions (see below), areas that are active bilaterally during normal speech production.”

“This mapping from desired sensory outcome to the appropriate motor action is an inverse kinematic transformation and is often referred to as an inverse model (e.g., Kawato, 1999; Wolpert & Kawato, 1998).”

“Evidence of mirror neurons in humans has implicated left precentral gyrus for grasping actions (Tai, Scherfler, Brooks, Sawamoto, & Castiello, 2004), and left opercular inferior frontal gyrus for finger movements (Iacoboni et al., 1999).”

“In the conversational speech task, additional higher-up processing had to occur prior to speech production, placing a greater demand on resource allocation (Bohland, Bullock, & Guenther, 2010; Tourville & Guenther, 2011; Van der Merwe, 1997).”

“…sounds generates somatosensory targets throughout speech motor development, representing proprioceptive and tactile sensations of the simultaneously generated sound (Guenther & Vladusich, 2012; Ito, Coppola, & Ostry, 2016; Skipper, Devlin, & Lametti, 2017; Tremblay, Shiller, & Ostry, 2003).”

“The cerebellum is another critical hub area of the brain (Akkal et al. 2007; Bostan et al. 2010, 2013; Brodal 1978; Glickstein et al. 1985; Hoover and Strick 1999; Hoshi et al. 2005; Kelly and Strick 2000, 2003; Leichnetz et al. 1984; Middleton and Strick 1994, 2001; Schmahmann and Pandya 1991, 1993, 1997; Snider and…”

“…goes beyond sensory feedback and speech comprehension and is more in line with proposals linking these regions to auditory and somato-sensory targets and predictive internal modeling-like mechanisms in speech production (e.g., Guenther et al., 2006; Hickok, 2012, 2014; Tourville & Guenther, 2011).”

“Theories of speech motor control include auditory feedback as crucial for both learning and maintaining such models (Parrell et al., 2019; Perkell et al., 1997; Tourville & Guenther, 2011).”

“It has been proposed that feedback from speech production to speech perception plays an important role in adjusting phonological representations (e.g., Houde & Nagarajan, 2011; MacDonald, Johnson, Forsythe, Plante, & Munhall, 2012; Tourville & Guenther, 2011; Villacorta, Perkell, & Guenther, 2007).”

“The axonal projections between the pre-SMA and the SMA would facilitate the interactions between the basal ganglia, motor, premotor cortex and the SMA to activate the “initiation map” in the SMA starting the pre-articulatory motor program at the appropriate timing.(26) Thus, the GODIVA model provides a relevant framew…”

“The feedforward system allows talkers to anticipate the sensory consequences of their speech production and gets more efficient with experience (Tourville and Guenther, 2011).”

“The cerebellum also plays important roles in the feedback control system which monitors speech output through auditory and somatosensory feedback to correct errors [39, 40].”

“Importantly, as these systems develop and become established, speech production is achieved primarily through a strong feedforward system, with less of a role of the auditory cortex and feedback control (Bohland and Guenther, 2006; Tourville and Guenther, 2011; Kearney and Guenther, 2019).”

“Understanding the causality of this brain-behavior relationship is important, not only allowing the refinement of the neural models of speech motor control (Hickok et al., 2011; Parrell et al., 2019; Tourville and Guenther, 2011) but also providing”

“To model this development computationally, a system can be equipped with a vocal tract model.”

“Indeed, the cerebellum is thought to regulate the timing among movements in a sequence (Spencer and Slocomb 2007; Tourville and Guenther 2011; Guenther 2016), particularly for behaviors that involve explicit temporal goals (Ivry et al.”

“In neuroanatomical approaches to modeling speech production, the left hemisphere is suggested to predominately host feedforward specifications, whereas the right hemisphere processes auditory feedback (Tourville and Guenther, 2011).”

“Moreover, the differential reliance on feed-forward and feedback mechanisms in CAS could have implications for differential processing in the left and right hemispheres (Tourville & Guenther, 2011).”

“…feedback in the fine-tuning of predictive, feed-forward control processes (e.g., the Directions-into-Velocities of Articulators, or DIVA, model, Tourville & Guenther, 2011; or the State Feedback Control, or SFC, model; Houde & Nagarajan, 2011; Houde & Chang, 2015; see also Tian & Poeppel 2010…”

“…an efference copy of the motor command is sent from motor planning areas to auditory and sensorimotor cortical regions to notify of impending feedback (Hickok, 2012; Hickok et al., 2011; Kearney & Guenther, 2019; Rauschecker, 2011; Rauschecker & Scott, 2009; Tourville & Guenther, 2011).”

“Computational models of speech production hypothesize that the cerebro-cerebellar projections contribute to the feedforward control of speech by mapping between the desired speech sound and its appropriate articulatory gesture (Tourville & Guenther, 2011).”

“, 2011; Hickok and Poeppel, 2007), whereas the DIVA model posits the involvement of bilateral pSTG in auditory feedback control of vocal production (Golfinopoulos et al., 2010; Tourville and Guenther, 2011).”

“The auditory feedback is incorporated in some computational models of speech acquisition and production (e.g., DIVA and GODVA models) to facilitate the training of the speech production process and simulate fine-grained articulatory movements (Tourville and Guenther, 2013).”

“Within the motor speech tradition, the main model is the ‘direc-tions into velocities of articulators’ (DIVA) model and its later version, ‘gradient order DIVA’ or GODIVA 3,38,39 .”

“Once the sensorimotor representation has been activated, both the feedback and feedforward systems send motor commands to the primary motor cortex (Tourville and Guenther, 2011; Guenther and Vladusich, 2012).”

“The IPMSP is an adaptation of models of speech processing from Levelt [43] and Guenther [40,41].”

“…the framework of a computationally sophisticated and neurally explicit model of motor speech control, DIVA (directions into velocities of articulators; Guenther, 1994, 1995; Tourville & Guenther, 2011), and its more recent version GODIVA (gradient order DIVA; Bohland, Bullock, & Guenther, 2010).”

“Applying the insights gained through models of speech motor control (Houde & Nagarajan, 2011; Parrell et al., 2019; Tourville & Guenther, 2011) to our results, the persistence of the Lombard effect may reflect an update of the feedforward system for motor control of voice.”

“D. Implications for models of speech production Our results have implications for current models of sensorimotor behaviour in speech production (Houde and Nagarajan, 2011; Tourville and Guenther, 2011; Parrell et al., 2019).”

“Coordination of laryngeal muscles in phonation depends upon motor planning, muscle activation, and feedback provided by auditory systems [1,3].”

“Finally, the finding that the medial frontal cortex, at a location corresponding to the supplementary motor area (SMA), responds more strongly to trained than to novel characters, is difficult to interpret in the present context: it is in line with the acknowledged role of this region in motor control of speech produc…”

“This finding supports the role of IFG as the speech sound map in the DIVA model; more specifically, left IFG as the input region for the entire speech–motor–control system, and thus the location for stored speech sound maps [Ghosh et al., 2008; Tourville and Guenther, 2011; Tourville et al., 2008].”

“Researchers have emphasized a transition from feedback-dominant to feedforward-dominant, driven by production experiences (Guenther, 2006; Guenther & Vladusich, 2012; Liu et al., 2010c; Scheerer et al., 2013).”

“Notably, there is evidence which points to some involvement of right-hemispheric analogues of typical language related areas in human speech (Tourville and Guenther, 2011; Zangwill, 1967).”

“Although the decrease in variability in older children has been associated with improved accuracy of productions [44,46,47], it is unlikely that differences in Fig 3.”

“…is a mismatch between the predicted auditory outcome and the actual auditory feedback, responses in the auditory regions are enhanced to encode the mismatch and inform vocal-motor regions to correct vocalization ( Hickok et al., 2011 ; Houde and Nagarajan, 2011 ; Tourville and Guenther, 2011 ).”

“Some of the most influential models of language production, e.g., DIVA and GODIVA (Guenther, 1995; see Tourville & Guenther, 2011, for a review) focus entirely on mapping a phonological form onto motor movements, thus ignoring any earlier (i.e., message-level) operations.”

“…auditory output is compared to incoming auditory feedback during 90 speech, and mismatches between the desired and actual auditory signals (e.g., formant 91 frequencies, pitch, voice quality) will result in “error signals” in auditory cortex (Guenther, 2006, 92 2016; Tourville & Guenther, 2011).”

“While we do not here explicitly relate this machine learning process to human learning, such maps could theoretically be learned during early speech acquisition, such as babbling [3].”

“While the propagation of sounds is the means through which linguistic information passes between speakers and listeners, most recent models of speech motor control [DIVA (1), FACTS (2), HSFC (3), Bayesian GEPPETO (4), and ACT (5)] posit that both auditory and somatosensory information is used during speech production…”

“…only monitors the smooth initiation and sequencing of speech sounds by use of feedback control processes, but also updates previously learned motor commands based on auditory information within the feedforward con-trol system (Brown et al., 2009; Hacker et al., 2012; Tourville & Guenther, 2011).”

“The present study was not designed to test models of speech production, but it does suggest that models, in particular those of speech motor con-trol such as the DIVA model (Tourville & Guenther, 2011), may be extended to incorporate mechanisms capable of generating between-talker variability.”

“40 Tourville and Guenther in 2011 proposed the Directions Into Velocities of Articulators (DIVA) model to describe the normal speech production neural network (Figure 2).”

“The Directions into velocities of articulators (DIVA) model (Guenther et al., 1998; Tourville and Guenther, 2010), developed by Guenther (2016), is a model of speech production and speech learning that focuses on feedforward and feedback mechanisms.”

“…form is a phonological string which can be executed as a motor plan, but at this level, too, there are claims about the nature of this string and the mapping between the abstract sounds and motor plan that fall outside the scope of this paper (Dell et al., 2007; Tourville and Guenther, 2011).”

“Note that two prominent models of speech motor control assume that motor-to-somatosensory and motor-to-auditory mappings occur in parallel rather than sequentially (Hickok, 2012a; Tourville & Guenther, 2011).”

“While we do not here explicitly relate this machine learning process to human learning, such maps could theoretically be learned during early speech acquisition, such as babbling [7].”

“Contemporary models of speech motor control, including the Directions Into Velocities of Articulators model (Tourville & Guenther, 2011), theorize that, during development of speech, the auditory control system relies primarily on a feedback control system to monitor and correct motor behaviors in…”

“Implicit or incidental learning is of particular importance to the acquisition of complex skills that require incremental fine-tuning of the perceptual-motor system and updating of internal models through repeated practice (Fitts, 1964; Tourville and Guenther, 2011).”

“Speech perception tasks engage a network of widely distributed brain regions, including primary and association auditory areas (e.g., bilateral superior temporal gyrus (STG) and superior temporal sulcus (STS)) involved in spectrotemporal and phonological processing, respectively.”

“Interpersonal differences in speaking rate were also incorporated into theoretical models of speech production, such as the DIVA model [19,20].”

“In a separate literature, however, work on neurocomputational models of speech motor control suggest that the quality of phonological representations hinges on the interaction between speech production (i.e. feed-forward) and speech perception (i.e. feedback) mechanisms (Tourville & Guenther, 2011).”