Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions

“…: response to an instruction cue pre- but closely resemble reward-predicting stimuli, despite corceding the reward-predicting trigger stimulus by a fixed interval of 1 s in rect behavioral discrimination (Schultz and Romo 1990).an instructed spatial reaching task (19 neurons) (Schultz et al. 1993).”

“…only to the onset of conditioned stimuli and not to their offset, even if stimuli are used whose offsets,Pavlovian or instrumental tasks, such as visual or audi- tory reaction time tasks, spatial delayed response or rather than their onsets, are valid predictors of reward (Schultz and Romo, 1990).”

“…other motivational signals in DA neurons that typically occur at longer latencies, take into account the precise identity of the stimulus, and are best correlated with considered behavioral actions such as decisions to approach or avoid (Schultz and Romo, 1990; Joshua et al., 2009a; Schultz, 2010).”

“, 1992) and the triggered task above (Romo and Schultz, 1990; Schultz and Romo, 1990), dopamine neuron responses transferred from reward delivery to sensory cues that predicted reward.”

“Reward-related stimuli, such as primary liquid and food rewards, and visual and auditory stimuli predicting such rewards elicit pure activations (Romo & Schultz 1990, Schultz & Romo 1990, Ljungberg et al 1992).”

“Although discriminating between reward-predicting CS and neutral stimuli, dopamine activations tend toward generalization (Schultz & Romo 1990, Mirenowicz & Schultz 1996, Waelti et al. 2001).”

“…of the stereotyped character of phasic responses of DA neurons to external stimuli seen before (Schultz, 1986; Romo and Schultz, 1990; Schultz and Romo, 1990) a standard time window procedure was developed in order to quantify and compare objectively the neuronal responses between…”

“…early in the acquisition lated with this motion, the absence of brain regions of lever pressing behavior (Schultz, 1986; Nishino et al., with time course changes specific for cocaine-induced 1987; Romo and Schultz, 1990; Schultz and Romo, 1990; craving is a matter to be interpreted with caution.”

“The incidence of activations to unrewarded, including aversive, stimuli increases with closer similarity and same sensory modality as rewarded stimuli (527, 597, 618).”

“In agreement with previous results (Schultz and Romo 1990), this suggests that responses were not related to the initiation of individual eye and arm movements but were determined by the properties of stimuli that, through the association with primary reward, had gained the capacity to elicit…”

“By contrast, the immediately following response component detects the true nature of the event through its graded activation with reward value [43] and its frequent depression with unrewarded and aversive stimuli [27,29,32,33] (Figure 1E).”

“Consistent with previous studies of reward coding (Schultz and Romo, 1990; Kawagoe et al., 2004; Roesch et al., 2007; Matsumoto and Hikosaka, 2007), we observed similar neural coding of behav-”

“…of reward, and with phasic inhibition of firing when an expected reward is not received (Fields et al., 2007; Schultz, 2006; Schultz et al., 1997; Schultz and Romo, 1990), consistent with roles in signaling a reward expectancy error and in cue-elicited approach behaviors (Fields et al., 2007;…”

“3(A) and (B) shows a feature that Schultz and Romo (1990), and also subsequent studies, have frequently noted for generalization responses.”

“In primates, neural firing in the ventral and dorsal striatum occurs immediately prior to approach behavior and continues until the reward has been obtained [Berns et al., 2001; Schultz and Romo, 1990].”

“Given the frequently observed generalization of dopamine responses to stimuli resembling reward predictors (Schultz and Romo, 1990; Ljungberg et al., 1991, 1992), dopamine neurons might receive separate inputs for the initial activation with poor reward discrimination and the later component that…”

“The latter initially interpreted the control of phasic firing in VTA as supporting “behavioral activation” [86] and “motivational arousal” [89].”

“They include classical conditioning (Graybiel et al. 1994), reinforcement learning (Dunnett and Iversen 1982; Schultz and Romo 1990; Robbins and Everitt 1996), and procedural or implicit learning (Butters et al. 1985; Squire 1986; Saint- Cyr et al. 1988; Knopman and Nissen 1991; Knowlton et al.…”

“Some dopamine neurons show depressions or activation-depression responses to non-reward-predicting neutral stimuli or even aversive stimuli when these are physically similar to rewarding stimuli and occur in similar contexts and close associations with them (Mirenowicz and Schultz 1996; Schultz and Romo 1990).”

“The firing pattern of DA cells appears to be a determinant of rewardmotivated behaviour (Nishino et al., 1987; Schultz and Romo, 1990).”

“42,46,69 In the case of successive reward-predicting stimuli, dopamine neurons are activated by the earliest stimulus.”

“…series of neurophysiological experiments, Schultz and his coworkers have recorded from midbrain DA neurons in monkeys during the performance of a variety of food reinforced behaviors (Schultz, 1986; Romo and Schultz, 1990; Schultz and Romo, 1990; Ljungberg et al., 1992; Schultz et al., 1993).”

“Naïve monkeys that had not received food in a particular setting showed activation of dopamine neurons only in response to food taste; however, after conditioning, dopaminergic activity began to precede reward delivery and eventually maximal activity was elicited by the conditioned stimuli that predicted the impending…”

“Based on these data, and other studies of substantia nigra pars compacta activity (Schultz & Romo 1990a,b; Ljungberg et al. 1992; Schultz et al. 1993), Schultz (Schultz et al. 1997, Schultz 1998) proposed that neurons of the pars compacta carry a signal encoding the difference in value between the…”

“The interpretation at the time was that dopamine neurons would participate in setting a state where actions would occur (Schultz & Romo 1990).”

“Dopamine neurons of the substantia nigra show specific responses that provide a neurophysiological basis for the involvement of dopamine systems in behavioral reactivity (Schultz and Romo 1990).”

“Thus, stimuli with similar sensory properties but with no actual association with reward often elicit phasic DA neuronal responses (Schultz and Romo, 1990).”

“…of dopaminergic systems which are heavily implicated in mediating the rewarding value of reinforcement and conditioned reinforcers (Wise 1978; Cador and Robbins 1991; Wilson et al. 1995), and in goal-directed behaviour (Wise and Rompre 1989; Schultz and Romo 1990; Robinson and Berridge 1993).”

“The first dopamine response component consists of a brief activation that begins with latencies of 60–90 ms and lasts 50–100 ms; it is unselective and arises even with motivationally neutral events, conditioned inhibitors and punishers (Steinfels et al. 1983; Schultz and Romo 1990; Mirenowicz and Schultz 1996; Horvitz…”

“As has been mentioned above, this control comes predominantly from the emotional system in the brain, the limbic system (Ljungberg et al., 1992; Mirenowicz & Schultz, 1994; Phillips, 1984; Schultz, 1998; Schultz & Romo, 1990; Schultz et al., 1995).”

“, 1983 ; Schultz and Romo , 1990 ) , whereas microdialysis investigations show strong associations between DA release and the motor demands of the task ( McCullough et al .”

“SNc neurons do not respond to active or passive body part movements, but respond to novel sensory stimuli and/or rewards (DeLong et al., 1983; Schultz and Romo, 1990).”

“, 1985) and are reported to fire bursts of spikes during locomotor tasks and when presented with reward-associated stimuli (Diana et al., 1989; Romo and Schultz, 1990; Schultz and Romo, 1990).”

“, 1996a,b), which resembles natural burst events as elicited by behaviorally significant stimuli (Schultz and Romo, 1990; Ljungberg et al., 1992; Schultz et al., 1993).”

“Generalization in the first component of dopamine responses occurs with neutral stimuli (Schultz and Romo, 1990; Waelti et al., 2001), aversive stimuli (Mirenowicz and Schultz, 1996; Joshua et al.”

“…neurons also regulate midbrain DA neurons, an operation that is consistent with behavioral state control, given that midbrain DA neuronal activity has been associated with behavioral activation and response to novel stimuli (Romo and Schultz, 1990; Schultz and Romo, 1990; Ljungberg et al., 1992).”

“DA cells are activated by auditory, visual, and somatosensory conditioned stimuli (Romo and Schultz, 1990; Schultz and Romo, 1990).”

“The cells show phasic increases in activity 50 /100 ms after the onset of a salient sensory event [59,105,111].”

“The ®ring rates of VTA dopaminergic neurons (Steinfels et al., 1983; Schultz & Romo, 1990; Ljungberg et al., 1992; Horvitz et al., 1997; for review see Schultz, 1998) and dopamine release in the NAS (Rebec et al., 1997; Saigusa et al., 1999) are increased by novel stimuli that evoke orientation and…”

“However, once the behavioral act is acquired, the DA system no longer responds to the learned stimulus event (Schultz, 1986; Schultz and Romo, 1990).”

“Dopaminergic neurons increase in firing rate when an animal touches food [78;79] and when liquid reward is administered to the animal outside of a behavioral context [77].”

“Midbrain dopamine neurons that project to the striatum respond to a large category of arousing events, including novel stimuli (Schultz and Romo, 1990; Legault and Wise, 2001), aversive stimuli (Young, 2004), and high intensity visual and auditory events (Steinfels et al.”

“In primates, presentation of different foods and sweet solutions, or visual and auditory stimuli paired previously with reward results in brief, phasic firing of DA neurons in the VTA (Romo & Schultz, 1990; Schultz & Romo, 1990; Ljungberget al., 1992).”

“, 1993; Gratton and Wise, 1994; Kiyatkin and Gratton, 1994) and with evidence from other sources indicating that increased DA cell activity is a correlate of reward expectancy (Nishino et al., 1987; Schultz and Romo, 1990; Ljungberg et al., 1992; Schultz et al., 1993).”

“The stimulus generalization responses are the composite response pattern of brief activation and subsequent suppression to sensory stimuli that are not associated with reward (Schultz and Romo, 1990; Horvitz et al., 1997; Waelti et al., 2001; Tobler et al., 2003).”

“, in the case of dopamine neurons in the ventral tegmentum (Schultz and Romo, 1990), pyramidal cells in various layers of the hippocampus or adjacent cortices (O’Keefe and Nadel, 1978), or face-recognizing neurons in the lateral lobe (Peirce et al.”

“Often these responses are strikingly biphasic (Schultz and Romo, 1990; Mirenowicz and Schultz, 1996; Waelti et al., 2001), as though a fast excitatory salience response was superimposed on a longer-latency inhibitory value response (Joshua et al.”