Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements

“In addition, the activation of dopamine neurons in the substantia nigra appears to depend on the motivational value of sensory stimuli that cue motor actions [55].”

“This pacemaking activity is believed to be important to maintaining ambient DA levels in regions that are innervated by these neurons, particularly the striatum (Romo and Schultz, 1990).”

“Therefore, auditory, visual, and somatosensory trained incitements stimulate DA cells (Romo and Schultz, 1990; Schultz and Romo, 1990), with bias toward auditory incitements at tremendously brief latencies, strictly programmed to the stimulus interval.”

“This pacemaking activity is believed to be important in maintaining ambient DA levels in regions that are innervated by these neurons, particularly the striatum (Romo and Schultz 1990).”

“R) task (Romo and Schultz, 1990; Ljungberg et al., 1992; Mirenowicz and Schultz, 1994), but declined with overtraining as the behavior became automatic (Ljungberg et al., 1992; Schultz, 1993).”

“umented in food-rewarded, lever-pressing tasks (Ljungberg et al., 1992; Romo and Schultz, 1990).”

“Indeed, both dopamine neuron firing (27, 28) and phasic mesolimbic dopamine release (29, 30) precede well-learned self-initiated actions, consistent with a backpropagating reward prediction error signal.”

“By contrast, dopamine neurons show phasic activations after reward-related events and certain attention-inducing stimuli of the somatosensory, visual, and auditory modality (5, 10, 12).”

“…stimuli eliciting orienting reactions, by primary reward during self-initiated movements in the absence of reward-predicting phasic stimuli (Romo and Schultz, 1990), and by conditioned, reward-predicting stimuli (Ljungberg et al., 1992; Schultz et al., 1993; Contreras-Vidal and Schultz…”

“…the activation of midbrain’s dopaminergic projections to fronto-parietal cortex and dorsal striatum that track successful and erroneous behaviors and the contingencies between the behaviors and rewards (Romo and Schultz, 1990; Gaspar et al., 1992; Kiyatkin, and Rebec, 1996; Fiorillo et al., 2003).”

“■”

“Although phasic activation of dopaminergic neurons is not correlated with specific motor movements, the types of stimuli that tend to promote burst firing and dopamine transients often induce behavioral activation (Nishino et al., 1987; Romo and Schultz, 1990; Robinson et al., 2002).”

“It was hypothesized that the increase in DA signaling is associated with the motivated go process or action (Romo and Schultz, 1990; Schultz and Romo, 1990; Ljungberg et al., 1992).”

“Dopamine is released following stimuli that predict reward (Romo and Schultz 1990), and noradrenergic neurons may respond to unpredicted more than predicted rewards (Aston-Jones et al. 1994; Foote et al. 1980).”

“The SNPC neurons are primarily involved in triggering motor reactions to stimuli of behavioural significance (Schultz 1986; Romo and Schultz 1990), and their degeneration is a well-known cause of Parkinson's disease.”

“…of similar changes in PFC and VTA DA unit activity during goal-directed behaviors (Ono et al., 1984; Inoue et al., 1985; Schultz, 1986; Nishino et al., 1987; Romo and Schultz, 1990; Schultz and Romo, 1990a,b; Ljungberg et al., 1992; Schultz et al., 1993; Kosobud et al., 1994; Watanabe, 1996).”

“, 2003), electrophysiology (Romo and Schultz, 1990; Schoenbaum et al., 1998; Tremblay and Schultz, 1999) and, more recently, optogenetics (Ferenczi et al.”

“Nigrostriatal DA neurons display two modes of discharge: a tonic background firing maintaining ambient dopamine levels in the striatum and a phasic firing (bursting activity) associated with brief high peaks of dopamine release, observed during appetitive learning and motivation-related behaviors (Gonon, 1988; Romo an…”

“Similarly, Schultz and his colleague (1990) reported that tactile sensation of a food reward elicited a phasic increase in DA activity when monkeys were required to find the reward in a covered box during self-initiated arm movements (Romo and Schultz, 1990).”

“It has indeed been shown that approximately 75% of dopamine neurons are synchronically activated when animals experience physical contact with a hidden food or liquid during an exploratory phase [200] and the figure is up to 55–70% for conditioned visual and auditory stimuli [213].”

“Keywords: Dopamine, prediction error, associative learning, blocking, latent inhibition, overshadowing, schizophrenia, reinforcement learning, incentive salience, motivated behavior, temporal difference algorithm, Rescorla–Wagner learning rule, psychosis”

“time limited, in the order of 100 ms) and in response to cues predicting upcoming reward delivery (Hikosaka et al. 2008; Romo and Schultz 1990; Schultz et al. 1997).”

“Because other striatal afferents show little correspondence to these striatal features (Peschanski et al., 1981; Steinfels et al., 1981; DeLong et al., 1983; Schultz et al., 1983; Schultz, 1986; Romo and Schultz, 1990; Ljungberg et al., 1992; Chudler and Dong, 1995), the transmission of detailed somatosensory and moto…”

“The LTCCs promote burst firing of these cells (Liu et al. 2007; Zhang et al. 2005), which is essential for salient stimuli (Chergui et al. J Neurophysiol • doi:10.1152/jn.00757.2013 • www.jn.org 1994; Romo and Schultz 1990; Schultz 2007, 2010; Tobler et al. 2005).”

“In the dopaminergic projection between the ventral tegmental area (VTA) and the prefrontal cortex (PFC), DA is released in bursts as a response to appetitive or novel stimuli (Romo and Schultz, 1990) or as background tonic activity (Seamans and Yang, 2004).”

“…DA neurons respond to cues associated with food delivery or the perception of food rewards, with the majority of responses excitatory and a minority inhibitory (Fiorillo et al., 2003; Mirenowicz and Schultz, 1994; Romo and Schultz, 1990; Satoh et al., 2003; Schultz, 1986; Schultz et al., 1993).”

“There are also minor activations or depressions before or during large reaching movements (Schultz et al., 1983; Schultz, 1986; Romo and Schultz, 1990), but neither with slightly different arm movements (DeLong et al., 1983; Ljungberg et al., 1992; Satoh et al., 2003) nor with licking or eye…”

“In monkeys, for example, it has been shown that the simple touch of a hidden food elicits phasic activation of dopaminergic neurons (19).”

“…Schultz, who pioneered single-unit recordings from the midbrain of awake and behaving monkeys, began recording the activity of dopaminergic neurons while monkeys underwent simple instrumental or Pavlovian conditioning ( Romo and Schultz, 1990 ; Ljungberg et al ., 1992 ; Schultz et al ., 1993 ).”

“Mesocorticolimbic DA function also maintains a prominent position in computational-error-correction models of learning (Romo and Schultz, 1990; Schultz, 2002; Schultz and Romo, 1990).”

“Data are from [4,9,10,26].”

“Similar to striatal neurons, primate dopamine neurons respond to reward expectation stimuli, reward receipt, and reward prediction error (Mirenowicz and Schultz, 1996; Schultz and Dickinson, 2000), separate from movements (Romo and Schultz, 1990).”

“1983), our observations are at variance with more recent studies that have suggested a direct link between tonic (Romo and Schultz 1990) or phasic (Bouret et al.”

“Even in monkeys, Romo and Schultz [91] reported gradual increases in the firing rates of some putative dopaminergic cells (12 out of 104 recorded) up to 1500 before onset of self-initiated arm movements to obtain food.”

“The pre-SMA notably becomes more active during motor preparation and action selection (Romo and Schultz, 1990).”

“…Department, Queens College/City University of New York, United States r t i c l e i n f o rticle history: eceived 1 May 2008 eceived in revised form 24 July 2008 ccepted 8 October 2008 vailable online 15 October 2008 a b s t r a c t Mechanisms underlying reward-related learning…”

“It may simply be that different aspects of TAN activity provide separate movement-control signals and feedback signals, much like has been suggested for the activity of midbrain dopaminergic neurons (Romo and Schultz 1990).”

“…modulated by neuromodulators (Seol et al., 2007; Pawlak and Kerr, 2008; Shen et al., 2008; Zhang et al., 2009; Pawlak et al., 2010; S. Cassenaer, and G. Laurent, personal communication), which are known to be key factors in memory retention (Kasamatsu and Pettigrew, 1976; Romo and Schultz, 1990).”

“Given that sustained increases in VTA dopamine neuron firing rate have been observed before self-initiated movement (Romo and Schultz 1990), we should emphasize that a limitation of our current task design prevents us from firmly concluding that the prestimulus activation is unrelated to motor…”

“…results are additionally consistent with reports that self-initiated action sequences are preceded by DA neuron firing (c.f., (Jin and Costa, 2010; Romo and Schultz, 1990) as well as mesolimbic DA release (Ko and Wanat, 2016; Phillips et al., 2003; Roitman et al., 2004; Stuber et al., 2005;…”

“…in the ventral tegmental area and substantia nigra of awake monkeys while they repeatedly performed an instrumental or Pavlovian conditioning task4 (Ljungberg, Apicella, & Schultz, 1992; Romo & Schultz, 1990; Schultz, Apicella, & Ljungberg, 1993; Schultz & Romo, 1990; Mirenowicz & Schultz, 1994).”

“This “offset hypothesis” is supported by data indicating that DA cells do not show responses that are time locked to the motor initiations gated by matricial MSPN outputs (DeLong et al., 1983; Romo and Schultz, 1990; Schultz, 1998).”

“Because modern land mammals have well-developed brain dopamine reward systems (Romo and Schultz, 1990; Ljungberg et al., 1992; Richardson and Gratton, 1996; Berridge and Robinson, 1998; Wise, 2008; Haber and Knutson, 2010; Schultz, 2010), and some make sounds connected with reward (Lilly, 1958; Brudzynski, 2007; Burgd…”

“However, repetitive exposure to the same stimulus facilitates a process of reward conditioning that enables the brain to evaluate future confrontations with the same set of factors it was conditioned to (Romo and Schultz 1990).”

“Both these behaviours are qualitatively identical to those exhibited by the original model and consistent with the experimental data (Romo and Schultz, 1990; Ljungberg et al., 1992) Brain networks: nested and=or overlapping 15”

“, 1992) and when conditioned stimuli predict their arrival (Romo and Schultz, 1990).”

“In fact, dopaminergic neurons possess a pacemaking activity of 2 to 4 Hz (Grace and Bunney, 1983) that maintains a proper concentration of dopamine (DA) in the striatum (Romo and Schultz, 1990).”

“The previous reports that dopaminergic neurons encode a prediction error in a process of instrumental conditioning which is learning a relationship between a stimulus, an action and a reinforcer determining the action (Romo and Schultz, 1990; Ljungberg et al., 1992; Schultz, 2000).”

“2–5 and references therein), measuring dynamic bias on a trial-by-trial basis is very challenging because bias is an internal state that we cannot directly observe.”

“Primate studies In the search for the neuronal organization of reward processing, Schultz and coworkers (Romo & Schultz, 1990; Schultz & Romo, 1990) used single-cell recordings in macaque monkeys.”

“These criteria were similar to those used by Schultz and coworkers (Aebischer & Schultz, 1984; Schultz, 1986; Romo & Schultz, 1990).”

“However, DA neurons in the food or fluid-deprived monkey show nonhabituating unconditioned phasic responses to food when the food is touched (Romo and Schultz, 1989, 1990), and juice when delivered directly into the mouth (Mirenowicz and Schultz, 1996; Hollerman and Schultz, 1998; Fiorillo et al.,…”

“Dopamine neurons in particular are activated by the rewarding characteristics of a wide range of somatosensory, visual, and auditory stimuli, and have been shown to distinguish between reward and nonreward objects (Romo and Schultz, 1990).”

“Decreased DA release, but unaltered DA uptake in Phe(+) mice Dopamine neurons display two modes of discharge: tonic background firing and phasic (burst) firing that is associated with motivation-related behaviors (Grace and Bunney 1984; Romo and Schultz 1990; Grace 1991; Ljungberg et al. 1992).”

“Physiological studies on primates have demonstrated that dopaminergic neurons in the nigrostriatal and mesolimbic systems display bursting patterns of depolarizing activity, instead of single spikes, in response to a reward, such as food.(37) When a stimulus is paired consistently with the delivery of a reward (as in…”