Visual and Nonvisual Contributions to Three-Dimensional Heading Selectivity in the Medial Superior Temporal Area

“It can be shown that such a network is capable of estimating an arbitrary function, given a sufficient number of hidden units (Bishop, 1995).”

“The overall heading estimate of the network, resulting from the activity of the output units, was defined by a population vector (Georgopoulos et al., 1986) and was computed as the vector sum of the responses of the 26 output units.”

“Both monkeys were also implanted with scleral coils for measuring eye movements in a magnetic field (Robinson, 1963).”

“As demonstrated previously by Zipser and Andersen (1988), the transformation of optic flow signals from an eye-centered to head-centered reference frame was implemented in the hidden layer of our model through modulation of hidden unit responses by eye position (i.”

“Responses from the MT could be further pooled by the MST for even higher-order motion processing, such as complex optic flow that simulates linear translation or rotation of the body as it navigates through the environment (Bremmer et al., 2010; Britten, 2008; Britten and Van Wezel, 2002; Duffy and Wurtz, 1995; Gu et…”

“The fact that MSTd is driven more strongly by visual than vestibular input (Gu et al., 2006; Takahashi et al., 2007; Chen et al., 2011b), and that the same is true of hMST (Smith et al., 2012), is also consistent with this interpretation.”

“We elected to have the motion trajectories follow a Gaussian velocity profile because: (1) It is a smooth, transient, and natural stimulus, (2) it evokes robust visual and vestibular responses in cortical multisensory neurons, such as those in the dorsal medial superior temporal (MSTd) and ventral intraparietal (VIP)…”

“Data are plotted in spherical coordinates using the Lambert scale for elevation (Gu et al., 2006) and color-coded as follows: data with elevation 45° are shown in red (CS) or blue (SS); those with elevation 45° are shown black.”

“We could then predict the movement of the platform accurately and use the predicted trajectory to precisely generate the visual stimuli as described below (Gu et al., 2006, 2007).”

“As observed previously, discrimination is best where tuning curves are steepest (Regan and Beverley, 1983, 1985; Wilson and Regan, 1984; Bradley et al., 1987; Vogels and Orban, 1990; Snowden et al., 1992; Hol and Treue, 2001; Purushothaman and Bradley, 2005; Gu et al., 2006; Chen et al., 2011c).”

“However, during heading estimation a vestibular overweighting as been reported consistently [2, 10, 11], whereas we did not observe such an overweighting with respect to optimal integration schemes.”

“On the other hand, there are multiple cortical areas that respond to vestibular stimuli, typically together with other sensory and motor signals (Grüsser et al., 1990; Bremmer et al., 2002; Fukushima et al., 2006; Klam and Graf, 2006; Chen et al., 2010, 2011a, 2011b, 2011c; Gu et al., 2006, 2016).”

“In this way, correlation-driven adaptation can naturally explain the alignment of preferred visual and vestibular selfmotion directions observed, for example, in area MSTd [32], a region that could potentially underlie crossmodal aftereffects.”

“Each contour map shows the Lambert cylindrical equal-area projection of the spherical data onto Cartesian coordinates (Gu et al., 2006).”

“Previous studies have reported that neurons in multiple cortical areas (e.g., MSTd, VIP, VPS) are tuned for heading, and tend to prefer either the same or opposite headings defined by optic flow and vestibular cues (Page and Duffy, 2003; Gu et al., 2006, 2008; Chen et al., 2011a, 2011b).”

“…MT-like patterns of activity can account for several essential response properties of MSTd neurons, such as 3D translation and rotation selectivity (Gu et al., 2006; Takahashi et al., 2007); tuning to radial, circular, and spiral motion patterns (Graziano et al., 1994; Lagae et al., 1994; Duffy…”

“Both cortical regions have the adequate functional properties to extract heading information from visual self-motion stimuli (Duffy and Wurtz, 1991; Lappe et al., 1996; Bremmer et al., 2002a,b, 2010; Gu et al., 2006, 2007, 2008, 2010, 2012; Chen et al., 2011a,b).”

“The animal was surgically implanted with a circular head holding device, a scleral coil for measuring eye movements, and a recording grid (Gu et al., 2006, 2008).”

“For nonuniform distributions, we also performed a multimodality test based on the kernel density estimate method (for details, see Gu et al. 2006; Takahashi et al. 2007).”

“Preferred directions in both the visual and the vestibular domain are not uniformly distributed, but bimodal with peaks at 90 to the left and to the right of straight ahead (106).”

“1992), an inseparable representation that can be useful for encoding self-motion and consistent with MST’s proposed role in heading perception (Britten and van Wezel 1998; Duffy and Wurtz 1995; Gu et al. 2006; Roy and Wurtz 1990).”

“Populations of neurons with opposite, rather than congruent, visual vestibular tuning [39, 40] are well-suited for detecting conflict between visual and vestibular signals.”

“These results contrast with those in VIP and MSTd neurons, where the majority of vestibular and visual neurons preferred lateral translation (Gu et al., 2006, 2010; Chen et al., 2011b).”

“Details have been described in previous publications (Chen et al. 2013a,b,c; Fetsch et al. 2007; Gu et al. 2006; Takahashi et al. 2007).”

“…has shown that both visual information and vestibular signals can mediate the neural responses in MST and VIP and the subsequent perception of motion, depending on the type of stimulus used and the statistics of eye and head movements (Britten 2008; Gu et al. 2008; Gu et al. 2007; Gu et al. 2006).”

“…until downstream multisensory areas, like the dorsal medial superior temporal area (MSTd), where neurons are tuned not only to optic flow and binocular disparity (Duffy &Wurtz, 1991a, 1991b; Gu et al., 2008; Roy & Wurtz, 1990) but also to vestibular cues (Gu et al., 2007, 2006; Page & Duffy, 2003).”

“…in the vestibular nuclei, and extend to the thalamus and further to the cerebral cortex are the neural substrate for conscious perception of spatial self-motion (Liedgren et al. 1976; Büttner and Lang 1979; Mergner et al. 1981, 1991; Grüsser et al. 1990; Bremmer et al. 2002; Gu et al. 2006).”

“This pattern further confirmed that we were in the correct areas (Gu et al. 2006).”

“Data were collected from five labyrinth-IN male rhesus monkeys (Macaca mulatta), which were chronically implanted with a plastic head-restraint ring and a guide tube platform, as described in detail in previous publications (Dickman and Angelaki 1999; Gu et al. 2006; Meng et al. 2005).”

“Surgical preparation and training have been described in detail in previous studies (Gu et al., 2006; Fetsch et al., 2007; Takahashi et al., 2007; Chen et al., 2013).”

“Platform motion and visual stimuli were updated at 60 Hz and precisely calibrated such that visual motion was synchronous with platform motion (Gu et al. 2006).”

“Both areas contain neurons that show selectivity for optic flow patterns and directional tuning for inertial motion in darkness (Tanaka et al., 1986; Duffy and Wurtz, 1991; Page and Duffy, 2003; Gu et al., 2006), and some of these neuronal populations are sensitive to congruent stimuli and others to incongruent stimul…”

“Additional details are provided in previous publications (Meng et al., 2005; Gu et al., 2006; Yakusheva et al., 2008).”

“…superior temporal area (MSTd), which is selective for heading from optic flow (Bradley, Maxwell, Andersen, Banks, & Shenoy, 1996; Duffy & Wurtz, 1995; Gu et al., 2006, 2008) and from vestibular cues (Bremmer, Kubischik, Pekel, Lappe, & Hoffmann, 1999; Duffy, 1998; Gu et al., 2006, 2007, 2008).”

“…the optic flow patterns used in previous studies on the monkey medial superior temporal area [Tanaka et al., 1989; Churchland and Lisberger, 2005; Gu et al., 2006] and on the cat lateral suprasylvian area [Li et al., 2000; Chen et al., 2004] because all the components of the surface are moved…”

“…is not known with certainty, there are neurons in the VIP and MSTd that respond to both visual and inertial headings (Bremmer et al. 2002; Gu et al. 2006, 2007, 2008; Page and Duffy 2003), suggesting that these areas play a role in visual-inertial integration (Fetsch et al. 2007, 2013)…”

“On the other hand, monkey multisensory regions known to be sensitive to visual motion, namely areas MST, VIP and VPS, are also implicated in multimodal estimate of heading by combining visual and vestibular cues to self-motion direction (Duffy, 1998; Bremmer et al., 1999; Schlack et al., 2002; Gu et al., 2006; 2008; C…”

“In particular, the vestibular system makes important contributions to our perception of self-motion, and various cortical regions in primates have been characterized with regard to vestibular heading selectivity (Gu et al., 2006, 2007; Chen et al., 2010, 2011a,b, 2013).”

“More recent studies have also found that some MSTd neurons respond optimally to a combination of vestibular and optic-flow stimulation and that the combination is important for sensing heading direction (Bremmer et al 1999; Gu et al 2006, 2007; Takahashi et at 2007; Telford et al 1995).”

“…regions associated with limb movement have not been identified, analogous modulation of optic flow neurons in MSTd by non-visual signals has been observed for pursuit eye movements (Newsome et al. 1988) and vestibular signals (Duffy, 1998; Bremmer, et al, 2001; Gu et al, 2006; Fetsch, et al, 2007).”

“Interestingly, it is notable that although the majority of sensory cortices contain a balanced mix of acceleration and velocity vestibular signals, MSTd neurons mainly have a velocity-like component, which is also true of its counterpart in the visual pathway [20, 32].”

“Animals were chronically implanted with a circular delrin ring for head restraint (Gu et al., 2006) and scleral search coils for monitoring eye position.”

“…generally similar stimulus-driven responses (Colby et al., 1993; Bremmer et al., 1997, 2002; Duffy, 1998; Page and Duffy, 2003; Zhang et al., 2004; Gu et al., 2006, 2007; Takahashi et al., 2007; Maciokas and Britten, 2010; Zhang and Britten, 2010; Chen et al., 2011b), but show major differences…”

“We further sorted the multisensory neurons into those with congruent and opposite vestibular and visual heading preferences ( Chen et al., 2011a ; Gu et al., 2006 ) with no observable differences ( Figure 5—figure supplement 1B ).”

“Although the superior temporal lobe has both congruent neurons and opposite neurons (Gu et al., 2006), the complexity of EEG data from superior temporal lobe reduced after repeated practice under congruent condition whereas the superior temporal lobe continuously played a key role in processing…”

“For eye coil implantation (Dickman & Angelaki, 2002; Gu et al. 2006), a pre-fabricated, sterilized eye coil was implanted beneath the conjunctiva of the eye.”

“Of course, in addition to visual, there are a number of other nonvisual cues that contribute to the perception of self-motion (Bradley et al. 1996; Cardin and Smith 2010; Fetsch et al. 2007; Gu et al. 2006; Kaminiarz et al. 2014; Royden et al. 1992).”

“The putative locus would likely be in relatively early motion-sensitive visual areas, as particularly the later motion area MST receives considerable input from the vestibular system (Chen et al., 2011; DeAngelis & Angelaki, 2012; Gu, Angelaki, & Deangelis, 2008; Gu et al., 2006).”

“One obvious source of information is the vestibular system (e.g., Fetsch et al. 2009; Gu et al. 2006; 2008; Hlavacka et al. 1992; for a review see Angelaki and Cullen 2008; Cullen 2012).”

“Because the motion platform has its own dynamics, we characterized the transfer function of the motion platform, as described previously (Gu et al., 2006), and we generated visual stimuli according to the predicted motion of the platform.”

“First, while the preferred motion directions of MSTd neurons cover the whole three-dimensional space [47], in the horizontal plane they are biased to the left – right axis, which is ideal for discriminating fine deviations from straight-forward heading from the computational perspective [47].”

“MST neurons respond selectively to visual optic flow patterns (Saito et al., 1986; Duffy and Wurtz, 1991, 1995) and are tuned to the vestibular components of self-motion (Page and Duffy, 2003; Gu et al., 2006; Angelaki et al., 2011).”

“It is also common to see another population of neurons with conflicting visual–vestibular tuning curves (‘conflict’ or ‘opposite’ neurons 19,21,87,91 ).”

“In such a representation, the mean firing rate in each trial was considered to represent the magnitude of a 3D vector whose direction was defined by the azimuth and elevation angles of the particular stimulus (Gu et al. 2006).”

“We confirmed the location of MT based on the pattern of gray matter and white matter encountered during electrode penetration before reaching MT as well as during the subsequent entry, after a short silent region (sulcus), into area MST, which had large, contralateral RFs covering a quadrant or half of the display (Gu…”

“visual and vestibular self-movement signals (Duffy 1998; Gu et al. 2006; Takahashi et al. 2007) and integrate those cues over time to represent the continually changing headings that define naturalistic paths through the environment (Froehler and Duffy 2002).”