Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey

“The dopaminergic (DA) neurons of the ventral tegmental area (VTA), together with its afferents to the nucleus accumbens (NAc) and to the prefrontal cortex (PFC), constitute a crucial neuronal system that processes natural rewards (Hollerman and Schultz, 1998; Schultz, 1986; Di Chiara et al, 1993; Schmidt, 1998; Wise,…”

“, 2002), and up to 100 Hz (task-related in primates) (Schultz, 1986)].”

“Neural activity in this region has been equated with dopamine release (Schultz, 1986) and identified as the source of the nigrostriatal, mesolimbic and mesocortical dopaminergic pathways.”

“These neurons are critical for the control of voluntary movement, which is regulated by the SNc projections to dorsal striatum via the nigostriatal pathway, and loss of these neurons leads to the impaired motor function observed in Parkinson’s disease (Frank 2005; Hikida et al. 2010; Hikosaka 2007; Jin & Costa 2010; K…”

“Furthermore, in salient behavioral situations midbrain dopamine neurones are reported to fire synchronously in bursts (Schultz, 1986).”

“During waking, presumed dopaminergic neurons tend to discharge in bursts of spikes and do so especially during behaviourally signi®cant and rewarding conditions (Schultz, 1986; Mirenowicz & Schultz, 1996).”

“Mesostriatal DA neurons in vivo demonstrate two modes of spontaneous neuronal discharge: single spiking at low, irregular frequencies (0.5‐ 8 Hz) or bursts with interspike rates of ;13‐20 Hz (or instantaneously higher, task-related) (Aghajanian and Bunney, 1973; Bunney et al., 1973; Grace and Bunney, 1983, 1984a,b; Sc…”

“However, when dopamine release was stimulated using multiple pulses in a manner that recapitulates burst-like firing patterns (usually 15–100Hz) thought to occur when rewarding or rewardpredictive stimuli are encountered (Schultz, 1986), nicotine instead increased dopamine release (Rice and Cragg, 2004; Zhang and Sulz…”

“In this context, midbrain DA neurons in the VTA and substantia nigra pars compacta (SNc) respond to alerting, activating, and reward-related stimuli (Trulson and Preussler, 1984; Schultz, 1986; Freeman and Bunney, 1987; Schultz et al., 1993).”

“…were either single pulses (1p) or trains of four pulses (4p) at frequencies ranging from “tonic” (1–10 Hz) to “phasic” burst frequencies ( 15– 40 Hz) of DA neurons that signal salient events in vivo (e.g., reward-predicting stimuli) (Schultz, 1986; Hyland et al., 2002; Bayer and Glimcher, 2005).”

“We tested whether the chronic sensors could be used to measure striatal dopamine changes in response to rewarding (1, 2, 13) (Fig.”

“Congruently, SNc DA cells are considered to be key for focusing attention on significant and rewarding stimuli, a requirement for the acquisition of new learned behaviors (Schultz, 1986; Schultz et al., 1997; Wise, 2009).”

“DA neurons in the VTA and substantia nigra respond to stimuli with alerting, activating, and rewardrelated properties (Schultz, 1986; Romo and Schultz, 1989; Ljungberg et al., 1992; Schultz et al., 1993).”

“Dopaminergic neurons fire bursts in vivo with intraburst frequencies that range from 14 to 30 Hz (Grace and Bunney, 1984; Schultz, 1986; Hollerman and Schultz, 1998; Hyland et al., 2002).”

“, 1980; Strecker and Jacobs, 1985; Kiyatkin, 1988) and during behavior (Schultz, 1986), the mean rate of activity remains relatively stable (Miller et al.”

“This is unexpected based on previous studies, which largely emphasized phasic activations to task events (but see Schultz, 1986; Fiorillo et al., 2003, 2008).”

“Dopamine is released in the NA in response to natural rewards such as food and water [241] and to conditioned stimuli linked to these rewards [205,242].”

“(Kimura et al., 1984; Schultz, 1986; Bayer et al., 2007).”

“Electrophysiological analysis of the activity of DA neurons during behavioral test have also demonstrated that these neurons are activated by very speci®c situations (Schultz, 1986; Mirenowicz and Schultz, 1996).”

“5 ms (Schultz, 1986), whereas those of confirmed GABAergic neurons typically are simpler and of shorter duration, less than 0.”

“In one of the first examinations of dopamine neurons in behaving primates, the animals would perform arm movements for small pieces of food hidden within a box (Schultz, 1986).”

“Thus we will assume that dopaminergic neurons in valuation, goal-directed, and habit systems can be mapped on a spectrum of dopaminergic neurons ranging from ventral tegmental area (VTA) including the valuation system to substantia nigra pars compacta (SNc) including the habit system.”

“The SNPC neurons are primarily involved in triggering motor reactions to stimuli of behavioural significance ( Schultz 1986; Romo and Schultz 1990), and their degeneration is a well-known cause of Parkinson's disease.”

“A well-described function of dopaminergic neurons in the VTA is in signalling a reward prediction error – the difference between expected and actual reward outcomes (Schultz, 1986).”

“These findings are congruent with electrophysiological evidence of similar changes in PFC and VTA DA unit activity during goal-directed behaviors (Ono et al., 1984; Inoue et al., 1985; Schultz, 1986; Nishino et al., 1987; Romo and Schultz, 1990; Schultz and Romo, 1990a,b; Ljungberg et al., 1992; Schultz et al., 1993;…”

“Burstfiring in the oldworldmonkey,Macacamulatta (rhesus macaque), is even higher and has beenmeasured as high as 36 Hz with meanburst rates of approximately 22Hz (Bayer et al., 2007;Matsumoto and Hikosaka, 2009; Hong and Hikosaka, 2014; Schultz, 1986; Schultz and Aebischer, 1983).”

“…selective opsin targeting, the authors were able to validate earlier proposed electrophysiological criteria for identifying midbrain DA neurons in extracellular recordings, including broader waveforms and lower baseline impulse rates compared to neighboring, non-dopaminergic neurons (Schultz 1986).”

“Substantial dopamine release is observed in inexperienced monkeys while food is in contact with their mouth, but dopamine ceases to be released as a response to food and becomes strongly related to a conditioned stimulus repeatedly associated with that food [8] [137] [213] [217].”

“Although a few studies have shown that the activity of some putative dopamine neurons may relate to specific behaviors (Kosobud, Harris, & Chapin, 1994; Schultz, 1986), the movement-related activity described here differs in two important ways.”

“Because other striatal afferents show little correspondence to these striatal features (Peschanski et al., 1981; Steinfels et al., 1981; DeLong et al., 1983; Schultz et al., 1983; Schultz, 1986; Romo and Schultz, 1990; Ljungberg et al., 1992; Chudler and Dong, 1995), the transmission of detailed somatosensory and moto…”

“…DA neurons respond to cues associated with food delivery or the perception of food rewards, with the majority of responses excitatory and a minority inhibitory (Fiorillo et al., 2003; Mirenowicz and Schultz, 1994; Romo and Schultz, 1990; Satoh et al., 2003; Schultz, 1986; Schultz et al., 1993).”

“Taken together, these findings suggest that the striatal TAN system is a possible neuronal substrate for rewarddriven behavior, working in concert with dopamine inputs from the midbrain (Schultz, 1986) in providing striatal circuitry with an error signal for the prediction of reward.”

“There are also minor activations or depressions before or during large reaching movements (Schultz et al., 1983; Schultz, 1986; Romo and Schultz, 1990), but neither with slightly different arm movements (DeLong et al., 1983; Ljungberg et al., 1992; Satoh et al., 2003) nor with licking or eye…”

“…[DA] o was compared in ventromedial versus dorsolateral regions of both the caudate and the putamen during pulse trains, over a range of frequencies (2–20 Hz) at which DA neurons may fire action potentials in situ (Grace and Bunney, 1983, 1984a,b; Schultz et al., 1983; Schultz, 1984, 1986).”

“We next applied a train of 5 pulses administered at 20 Hz, a stimulation pattern within the reported physiological range of phasic dopamine neuron firing (Grace and Bunney 1984; Hyland et al. 2002; Schultz 1986).”

“The two neuromodulators, dopamine (DA) and acetylcholine (ACh), have been linked to the selection of relevant and rewardpredicting sensory stimuli (Schultz, 1986; Aosaki et al., 1994a; Berridge and Robinson, 1998).”

“largely upon analysis of dopamine antagonist effects on reinforced learning, and preceded the pioneering work by Schultz and colleagues demonstrating that phasic midbrain DA neurons respond to reward and reward-paired stimuli [130,134].”

“This standard firing pattern is interrupted by phasic bursting activity (high-frequency firing) that accompanies salient, novel, or intense stimuli (Schultz 1986, 2002; Hyland et al. 2002; Rice and Cragg 2004; Zhang and Sulzer 2004; Cragg 2006; Heien and Wightman 2006).”

“To probe differences in dopamine release magnitude, we used 5 pulse stimulations and varied stimulation frequency to mimic the within-burst firing rates of dopamine neurons (Schultz, 1986; Hyland et al., 2002).”

“The mesocorticolimbic DA system has been the main target of studies characterizing the neural substrates mediating the role of environmental control of drug- or food-motivated behaviors [e.g., see (Gratton and Wise, 1994; Phillips et al., 1993; Schultz, 1986)].”

“This form of learning is implemented by dopaminergic neurons in the midbrain that change their level of activity when reward itself or reward prediction errors are encountered (Schultz 1986; Schultz et al. 1997; Pan et al. 2005; Pessiglione et al. 2006).”

“If novel stimuli are not reinforced, burst responses of dopaminergic neurons gradually disappear (Schultz, 1986).”

“In the monkey, Schultz and colleagues found that midbrain dopamine neurons fired in response to primary rewards such as food and liquid, but also to conditioned rewards, such as a light, picture, or sound that was predictive of a primary reward (Schultz, 1986; Schultz et al., 1993, 1997).”

“Phasic activity of DA neurons in the ventral tegmental area (VTA) signals the difference between reward receipt and the expected value of that reward (Glimcher, 2011; Schultz, 1986), termed reward prediction error (RPE) (Rescorla and Wagner, 1972).”

“…Psychology Department, Queens College/City University of New York, United States r t i c l e i n f o rticle history: eceived 1 May 2008 eceived in revised form 24 July 2008 ccepted 8 October 2008 vailable online 15 October 2008 a b s t r a c t Mechanisms underlying reward-related learning…”

“No dopaminergic-like patterns of cell activity (Schultz, 1986b) or waveforms were observed, which is consistent with a conclusion that we sampled neurons in SNpr rather than dopamine neurons in pars compacta.”

“SN pars compacta neurons phasically respond to reward-related sensory cues or to the reward itself, whereas striatal neurons increase their firing during the expectancy of forthcoming stimuli in behavioral paradigms, including the delivery of the reward (Schultz, 1986; Schultz et al., 1992; Schultz, 1997; Schultz, 201…”

“Moreover, these inputs appear to potently innervate VTA dopamine (DA) neurons (Georges & Aston-Jones, 2001, 2002) leading to their phasic excitation, which is a neural process fundamental to motivated behavior (Adamantidis et al., 2011; Schultz, 1986; Wanat et al., 2009).”

“As a way to assess this relationship, he used single-cell recordings of dopaminergic neurons in awake monkeys while they were performing reaching movements for food reward in response to auditory or visual stimuli (Schultz et al., 1983; Schultz, 1986).”

“We injected mice with quinpirole, which activates D2-like DA autoreceptors on the presynaptic terminals of DA neurons and presumably reduces the firing rate of DA but not GABA neurons [33,34] (Fig.”

“Neurons were classified as dopamine neurons based on a combination of previously used criteria (Schultz, 1986; Satoh et al., 2003; Kawagoe et al., 2004).”

“For reasons of comparison, the stimulus components of the opening door were identical to those used in another study in which the activity of DA neurons in intact, behaving monkeys was recorded (Schultz 1986).”

“…potentials at higher frequencies (2–5 spikes at ~ 15–40 Hz or higher) in response to events of motivational significance (e.g. reward predictions) (Schultz, 1986, 2002; Hyland et al., 2002; Brischoux et al., 2009; Matsumoto & Hikosaka, 2009). nAChRs on DA axons strongly govern the sensitivity…”

“In addition, the striatum is known to receive phasic and widespread waves of dopamine in response to the ®rst signal that predicts a coming reward, or to the reward itself if unexpected (Schultz, 1986; Schultz et al., 1993).”

“Moreover, phasic firing in primates is known to reach up to 100 Hz in frequency (Schultz, 1986).”

“putamen serves multiple functions, especially behaviours associated with motor function and reward.(19,20) Significant movement pathology, including impaired gait,(3) as well as deficient sense of reward, impaired, for example, in depression are affected in HF.”

“Electrophysiological studies in behaving animals (Schultz 1986, 1998; Kiaytkin 1988; Schultz and Romo 1990; Miller et al. 1991; Schultz et al. 1992) have shown that DA cells switch to burst firing following the occurrence of salient, novel, or reinforcing (unconditioned and conditioned) stimuli.”