Dopamine, Time, and Impulsivity in Humans

“These results show that the characteristic decrease in activity of these regions as rewards become more delayed (or increase as they become temporally closer) (McClure et al., 2004; Tanaka et al., 2004; Kable and Glimcher, 2007; Pine et al., 2009) (see also supplemental Results for placebo, available at www.”

“However, temporal (or choice) impulsivity— the preference for smaller–sooner over larger–later rewards, due to excessive discounting of future rewards (Ainslie, 1975; Evenden, 1999; Ho et al., 1999; Cardinal et al., 2004)—is much harder to account for in terms of learning, although it remains an…”

“This indicates that, unlike the choice outcome, dopamine manipulation did not influence the amount of time given to weigh a decision, or ability to “hold your horses,” and corroborates the suggestion that impulsivity is not a unitary construct (Evenden, 1999; Ho et al., 1999; Winstanley et al., 2004a; Dalley et al., 2…”

“Subjective effects were analyzed by comparing changes in the three factors identified by Bond and Lader (1974), namely, alertness, contentedness, and calmness, relative to the change in scores observed in the placebo condition.”

“…activity of these regions as rewards become more delayed (or increase as they become temporally closer) (McClure et al., 2004; Tanaka et al., 2004; Kable and Glimcher, 2007; Pine et al., 2009) (see also supplemental Results for placebo, available at www.jneurosci.org as supplemental material) is…”

“D can be thought of as the discount factor—the delay-dependent factor (between 0 and 1) by which the utility is discounted in a standard hyperbolic fashion (Mazur, 1987).”

“…syndrome, addiction, and ADHD, all of which are associated with hyperdopaminergic states caused by striatal dopamine flooding or sensitization (Solanto, 1998, 2002; Seeman and Madras, 2002; Berridge, 2007; Robinson and Berridge, 2008; Dagher and Robbins, 2009; O’Sullivan et al., 2009).”

“located) [90,91] and of the PFC, including OFC [92], and is sensitive to DA manipulations [93].”

“Interestingly, delay discounting seems to depend on the function of the ventral striatum (104) and of the PFC, including OFC (105) and its connections to the NAc (106), and is sensitive to DA manipulations (107).”

“sensitive to DA manipulations (Pine et al. 2010).”

“Hence, these data suggest that D2/D3 receptor signalling contributes to at least some aspects of impulsivity, consistent with a report that levodopa increased in delay discounting in healthy volunteers, an effect evident in every participant (Pine et al., 2010).”

“Also, hyperbolic discounting or impulsive choice for more immediate rewards in humans is related to dopamine activation (99).”

“It has been revealed (in ‘normal’ adults) that excess dopamine levels (by the administration of levodopa versus placebo) can selectively increase choice towards more immediate rewards, by influencing the rate of the discounting function, such that increases in delay appear to become intolerable (Pine et al., 2010).”

“Third, findings from nonhuman's delay discounting of drug reinforcers are heterogeneous; they do not suggest that drug administration, in and of itself, is sufficient to change this discounting (e.g., Harty et al. 2011; but see Pine et al. 2010).”

“A second framework involves dopamine’s more immediate effects on behavior (Frank et al., 2004; Niv et al., 2007; Robbins and Everitt, 2007; Salamone et al., 2007; Lex and Hauber, 2008; Palmiter, 2008; Sharot et al., 2009; Pine et al., 2010; Jocham et al., 2011; Guitart-Masip et al., 2012; Shiner et al., 2012; Norbury…”

“For the fMRI analysis,we estimated one single set of canonicalmodel parameters (αc+, αc−, αu+, αu−, τ) for all participants, similarly as in previous studies (e.g., Pine et al., 2010; Seymour et al., 2012; Voon et al., 2010).”

“Furthermore, administration of the DA precursor L-DOPA increases impulsive choice (Pine et al., 2010) in human volunteers, but does not affect SST or go/no-go performance (Overtoom et al., 2003; Hershey et al., 2004).”

“, 2010), converging evidence indicates that representations in the ventral striatum are modulated by whether outcomes are immediate versus delayed, and it has been shown repeatedly that the ventral striatum exhibits greater responses to the choice of sooner versus later outcomes (McClure et al., 2004; Kable and Glimch…”

“Dopaminergic medications can increase impulsivity in humans (Pine et al., 2010) and rodents (Logue et al.”

“Previous imaging studies have shown that amygdala activation was associated with individual differences in discounting, such that increased amygdala activity predicted more impulsive behavior (Hoffman et al., 2008; Pine et al., 2010).”

“Not surprisingly, delay discounting seems to depend on the function of the NAc (Gregorios-Pippas et al., 2009) and the PFC, including lateral OFC (Bjork et al., 2009), and is sensitive to DA manipulations (Pine et al., 2010).”

“Figure 2B demonstrates the effect of systemic dopamine agonists on discounting, adapted from Pine et al. (2010).”

“These deficits arise from dysregulation within frontal/basal ganglia circuitry that is often linked with dopamine dysfunction (Volkow et al., 2009; Pine et al., 2010).”

“Elevated dopamine function, in comparison, increases the ability of reward related cues to guide behavioral choices [65], diminishes the ability to differentiate between high and low value rewards [71], and induces steeper temporal discounting, a form of impulsivity defined by preference for immediately available smal…”

“In PD patients, administration of L-dopa correlated with greater attenuation of the dorsal striatum, insula, subgenual cingulate, and lateral orbitofrontal cortices for delayed relative to more immediate rewards, paralleling the behavioural result of increased impulsivity and temporal discounting relative to placebo […”

“…134; 1447–1463 P. A. MacDonald et al. D ow nloaded from https://academ ic.oup.com /brain/article-abstract/134/5/1447/286814 by U niversity of W estern O ntario user on 05 D ecem ber 2019 discounting) are combined to enact decisions (Pine et al., 2009, 2010; Ding and Gold, 2010; Cai et al., 2011).”

“The dopaminergic system is also suggested to play a role in impulsivity and delay of gratification [24],[39],[40], and some evidence suggests a potential role of”

“After correction for attenuation, random and scatter events, images were reconstructed using a Hanning filter of 2.0 mm into a volume of 47 slices, with a center-to-center distance of 3.125 mm and a pixel size of 2.02 mm×2.02 mm.”

“…incorporated into the construction of its ultimate value, suggesting a mechanism through which DA may influence human choice and temporal discounting, and traits such as impulsiveness that may account for several behavioral disorders associated with a hyperfunctioning DA system (Pine et al, 2010).”

“Alternatively, they may discount modestly, but exhibit steep diminishing marginal utility (Ho et al., 1999; Pine et al., 2009, 2010).”

“Second, human pharmacology studies suggest that DA modulates Now bias, albeit with inconsistent results (de Wit et al., 2002; Acheson and de Wit, 2008; Hamidovic et al., 2008; de Wit, 2009; Pine et al., 2010), although these studies have not accounted for intrinsic variations in DA signaling that could interact with p…”

“Both models rely on a softmax function for producing choice probabilities based on the values of the options including an inverse temperature (beta) parameter that determines the stochasticity of choice (see Pine et al., 2009, 2010 for details).”

“Evidence that some brain valuation sites do not respond to delayed rewards has been contradicted by research showing that all valuation sites respond to delayed prospects and discount them at the same rate (Kable and Glimcher 2007; Pine et al. 2010 ).”

“…broader range of functions (Berridge, 2007; Zhang et al , 2009) that are subject of much recent work (Palmiter, 2008; Rigoli et al , 2016a; Sharot, et al , 2009a; Pine et al , 2010; Jocham et al , 2011; Guitart-Masip et al , 2012; Shiner et al , 2012; Wunderlich et al , 2012; Norbury et al , 2013).”

“More recently, maximum likelihood estimation (MLE) has become a popular alternative to analyses based on indifference points (McClure et al. 2004; McClure et al. 2007; Pine et al. 2009; Pine et al. 2010).”

“Alternatively, he may discount modestly, but exhibits steep diminishing marginal utility such that the utility of the larger amount is only slightly larger than that of the smaller (Ho et al., 1999; Pine et al., 2009; Pine et al., 2010).”

“This choice function is frequently applied in computational modelling of reinforcement learning [60,61] and decisionmaking, including delay discounting [34,41,50,51,62,63] to convert value differences between decision options into choice probabilities.”

“A variety of evidence links delay-discounting to dopamine (DA) and DA-modulated frontostriatal circuits (Adriani et al. 2009; Altamirano et al. 2011; Boettiger et al. 2007; Doya 2008; Kobayashi and Schultz 2008; Lee et al. 2009; Mitchell et al. 2007; Paloyelis et al. 2010; Pine et al. 2010).”

“However, this seems unlikely since similar doses of L-DOPA given to healthy human volunteers induce behavioural effects [77– 79] and increase striatal DA synthesis [80].”

“…to THC may elicit persistent modifications in impulsivity is particularly interesting, in view of the proposed role of impulsivity in drug addiction and of the possible involvement of NAc shell DA in impulsivity (Basar et al. 2010; Besson et al. 2010; Pine et al. 2010; Sesia et al. 2010).”

“, 2009) and a correlation between striatal BOLD signals and the hyperbolic discount functions (Pine et al., 2009; Pine et al., 2010).”

“…of levodopa to healthy volunteers indeed increased impulsive choice of a small, immediate reward over a large, delayed reward by enhancing the diminishing influence of increasing delay on reward value (temporal discounting) and its corresponding neural re-presentation in the striatum [58].”

“Finally, a single study found effects opposite to our hypothesis -- increased delay discounting when augmenting DA using L-DOPA (150mg) (Pine et al., 2010).”

“Administration of the DA precursor L-dopa increases delayed reward discounting in healthy individuals [65].”

“2010 (95) 14 (6) Discounted utility 9 × Sasse et al.”

“However, the same dose of haloperidol did not affect performance on a slot machine game (Zack and Poulos, 2007), nor did a smaller dose (1.5 mg/kg, p.o.) affect rates of delay discounting in healthy adults (Pine et al., 2010).”

“Moreover, some data revealed a specific role for dopamine in controlling the relationship between the timing of future rewards and their subjective value [78].”

“Related to this, administration of dopamine antagonists to healthy participants increases the degree of temporal discounting, i.e., the relative value of delayed rewards decreases, leading to impulsive behavior (Pine et al., 2010).”

“Levodopa, a precursor to DA in healthy humans, increases impulsive choice (Pine et al. 2010) and increases delay aversion in patients with PD (Cools et al.”

“…recent study demonstrated that administration of L-Dopa to healthy volunteers engaged in an inter-temporal choice task enhanced VS BOLD activation to immediate rewards but attenuated VS BOLD activation to delayed rewards—an effect paralleled by an increase in delay discounting (Pine et al., 2010).”

“2010) and by pharmacological manipulations that heavily modulate subcortical DA in humans (Acheson and de Wit 2008; de Wit 2009; de Wit et al. 2002; Hamidovic et al. 2008; Pine et al. 2010).”

“[149] Furthermore, administration of the DA precursor L-3,4-dihydroxyphenylalanine (L-DOPA) can induce impulsive responding in healthy subjects[150] and in Parkinson’s disease patients.”

“However, other dopaminergic agents such as levodopa and pramipexole increase impulsivity on delay discounting and gambling tasks (24, 25).”

“Striatal dopamine is crucial for inhibitory control, updating working memory and decision-making processes (Cools et al., 2007; Hershey et al., 2004; Mehta et al., 2005; Pine et al., 2010).”

“Multiple studies have shown that areas of the brain with dopaminergic innervation are involved in temporal discounting preference (Kobayashi and Schultz, 2008; Pine et al., 2010).”

“We speculate that oxytocin might have reduced impulsiveness via interactions with the dopaminergic system as delay of gratification has been related to dopaminergic activity (Pine et al., 2010; Weber et al., 2016), whereby blocking dopaminergic neurotransmission increases delay of gratification similar to our current…”

“…that the effects of dopaminergic stimulation with pramipexole can significantly increase impulsive responding in non-lesioned rats (Madden et al., 2010; Johnson et al., 2011; Rokosik and Napier, 2012; Engeln et al., 2016), and that LD can increase impulsivity in healthy humans (Pine et al., 2010).”

“Often, the predictions of different accounts are even in apparent conflict with each other: for example, while some accounts claim that dopamine increases the tolerance for delay costs (8,9), others posit that dopamine enhances the preference for immediate rewards (10,11).”

“Commensurate with this hypothesis, systemic administration in humans of the dopamine precursor l-Dopa increases discount rates (Pine et al., 2010), although potentially countervailing evidence is that decreasing dopamine transmission in rats by administration of haloperidol (Denk et al., 2005) or…”

“…been linked to changes in striatal activity and to increases in impulsive-compulsive behavior and risk-taking choices (St Onge and Floresco 2009; Pine et al. 2010; O’Sullivan et al. 2011; Fineberg et al. 2010), suggesting more specific involvement of dopamine in appetitive reward value and in…”