Dopamine, Time, and Impulsivity in Humans

“These results show that the characteristic decrease in activity of these regions as rewards become more delayed (or increase as they become temporally closer) (McClure et al., 2004; Tanaka et al., 2004; Kable and Glimcher, 2007; Pine et al., 2009) (see also supplemental Results for placebo, available at www.”

“However, temporal (or choice) impulsivity— the preference for smaller–sooner over larger–later rewards, due to excessive discounting of future rewards (Ainslie, 1975; Evenden, 1999; Ho et al., 1999; Cardinal et al., 2004)—is much harder to account for in terms of learning, although it remains an…”

“D can be thought of as the discount factor—the delay-dependent factor (between 0 and 1) by which the utility is discounted in a standard hyperbolic fashion (Mazur, 1987).”

“This indicates that, unlike the choice outcome, dopamine manipulation did not influence the amount of time given to weigh a decision, or ability to “hold your horses,” and corroborates the suggestion that impulsivity is not a unitary construct (Evenden, 1999; Ho et al., 1999; Winstanley et al., 2004a; Dalley et al., 2…”

“Subjective effects were analyzed by comparing changes in the three factors identified by Bond and Lader (1974), namely, alertness, contentedness, and calmness, relative to the change in scores observed in the placebo condition.”

“…activity of these regions as rewards become more delayed (or increase as they become temporally closer) (McClure et al., 2004; Tanaka et al., 2004; Kable and Glimcher, 2007; Pine et al., 2009) (see also supplemental Results for placebo, available at www.jneurosci.org as supplemental material) is…”

“…syndrome, addiction, and ADHD, all of which are associated with hyperdopaminergic states caused by striatal dopamine flooding or sensitization (Solanto, 1998, 2002; Seeman and Madras, 2002; Berridge, 2007; Robinson and Berridge, 2008; Dagher and Robbins, 2009; O’Sullivan et al., 2009).”

“…but did not impact on deliberation—“holding your horses” (Frank et al., 2007)—that occurs when options are closely valued, engendering decision– conflict (Botvinick, 2007; Pochon et al., 2008) also related to reflection or preparation impulsiveness (Evenden, 1999; Clark et al., 2006).”

“DOI:10.1523/JNEUROSCI.6028-09.2010 Copyright © 2010 the authors 0270-6474/10/308888-09$15.00/0 effect on the rate of slowing down engendered by decision– conflict (Frank et al., 2007; Pochon et al., 2008) to distinguish global from discrete influences on impulsivity.”

“The characteristic loss of self-control and impulsivity associated with aberrant dopamine function is exemplified by disorders such as addiction, attention-deficit/hyperactivity disorder (ADHD), and dopamine dysregulation syndrome (Winstanley et al., 2006; Dagher and Robbins, 2009; O’Sullivan et al., 2009).”

“We used a sequence designed to optimize functional sensitivity in the orbitofrontal cortex (Deichmann et al., 2003).”

“D can be thought of as the discount factor—the delay-dependent factor (between 0 and 1) by which the utility is discounted in a standard hyperbolic fashion (Mazur, 1987).”

“…via presynaptic effects, which may explain its dose-dependent effects in many previous studies as well as its therapeutic efficacy (in moderate doses) in a putatively hyperdopaminergic ADHD (Seeman and Madras, 1998, 2002; Solanto, 1998, 2002; Solanto et al., 2001; de Wit et al., 2002).”

“, 2008), whereas others demonstrate the opposite, a dosedependent effect, or no effect (Logue et al., 1992; Charrier and Thiébot, 1996; Evenden and Ryan, 1996; Richards et al., 1999; Cardinal et al., 2000; Isles et al., 2003; Helms et al., 2006; Bizot et al., 2007; Floresco et al., 2008).”

“These studies could be confounded by the concomitant release of serotonin (Kuczenski and Segal, 1997), which is also implicated in the modulation of intertemporal choice.”

“…intertemporal choice indicate that addicts and a subgroup of ADHD patients appear to have abnormally high temporal discount rates, strongly preferring smaller–sooner rewards (Sagvolden and Sergeant, 1998; Bickel and Marsch, 2001; Solanto et al., 2001; Winstanley et al., 2006; Bickel et al., 2007).”

“In principle, some of the aforementioned deficits can be related to dopaminergic effects by way of dopamine’s established role in reward learning (Redish, 2004; Frank et al., 2007; Dagher and Robbins, 2009).”

“These include haloperidol’s nonspecific and widespread pharmacological effects or dosage—some studies indicate haloperidol may paradoxically boost dopamine in small doses, due to presynaptic effects on the D2 autoreceptor (Frank and O’Reilly, 2006).”

“…the choice outcome, dopamine manipulation did not influence the amount of time given to weigh a decision, or ability to “hold your horses,” and corroborates the suggestion that impulsivity is not a unitary construct (Evenden, 1999; Ho et al., 1999; Winstanley et al., 2004a; Dalley et al., 2008).”

“Thus our findings reveal a novel mechanism by which dopamine influences human decision-making that can account for behavioral aberrations associated with a hyperfunctioning dopamine system.”

“…et al., 1999; Cardinal et al., 2000; Wade et al., 2000; Isles et al., 2003; Winstanley et al., 2003; van Gaalen et al., 2006; Bizot et al., 2007; Floresco et al., 2008), whereas others demonstrate the opposite, a dosedependent effect, or no effect (Logue et al., 1992; Charrier and Thiébot,…”

“…Pavlovian impulsivity, where responses associated with primary, innate values form a simple, evolutionarily specified action set operating alongside, and sometimes in competition with, other control mechanisms, such as habit-based and goal-directed action (Dayan et al., 2006; Seymour et al., 2009).”

“, 2008) also related to reflection or preparation impulsiveness (Evenden, 1999; Clark et al., 2006).”

“, 2008), whereas others demonstrate the opposite, a dosedependent effect, or no effect (Logue et al., 1992; Charrier and Thiébot, 1996; Evenden and Ryan, 1996; Richards et al., 1999; Cardinal et al., 2000; Isles et al., 2003; Helms et al., 2006; Bizot et al., 2007; Floresco et al., 2008).”

“The characteristic loss of self-control and impulsivity associated with aberrant dopamine function is exemplified by disorders such as addiction, attention-deficit/hyperactivity disorder (ADHD), and dopamine dysregulation syndrome (Winstanley et al., 2006; Dagher and Robbins, 2009; O’Sullivan et al., 2009).”

“…attenuation leads to an increase (Richards et al., 1999; Cardinal et al., 2000; Wade et al., 2000; Isles et al., 2003; Winstanley et al., 2003; van Gaalen et al., 2006; Bizot et al., 2007; Floresco et al., 2008), whereas others demonstrate the opposite, a dosedependent effect, or no effect…”

“…enhancement leads to a decrease in impulsive choice or that dopamine attenuation leads to an increase (Richards et al., 1999; Cardinal et al., 2000; Wade et al., 2000; Isles et al., 2003; Winstanley et al., 2003; van Gaalen et al., 2006; Bizot et al., 2007; Floresco et al., 2008), whereas others…”

“…(PIT), a phenomenon dependent on connectivity between amygdala and striatum (Cardinal et al., 2002; Seymour and Dolan, 2008), and whose expression is known to be modulated by dopamine (Dickinson et al., 2000; Lex and Hauber, 2008), appetitive Pavlovian values increase responding for rewards.”

“Notably, individual susceptibility to this influence correlates with amygdala activity (Talmi et al., 2008), suggesting that the amygdala might modulate the extent to which primary conditioned and unconditioned reward values influence instrumental (habitbased and goal-directed) choice.”

“Dopamine is known to have a dominant effect on primitive reward behaviors such as approach and consummation (Parkinson et al., 2002).”

“…preference for smaller–sooner over larger–later rewards, due to excessive discounting of future rewards (Ainslie, 1975; Evenden, 1999; Ho et al., 1999; Cardinal et al., 2004)—is much harder to account for in terms of learning, although it remains an important feature of putative dopaminergic…”

“, 2000) and that destruction of serotonergic neurons can block the effects of amphetamine (Winstanley et al., 2003).”

“…via presynaptic effects, which may explain its dose-dependent effects in many previous studies as well as its therapeutic efficacy (in moderate doses) in a putatively hyperdopaminergic ADHD (Seeman and Madras, 1998, 2002; Solanto, 1998, 2002; Solanto et al., 2001; de Wit et al., 2002).”

“The greater K or r, the more the individual is likely to choose the sooner option and therefore the more impulsive is the individual (Ho et al., 1999; Pine et al., 2009).”

“…et al., 2008), whereas others demonstrate the opposite, a dosedependent effect, or no effect (Logue et al., 1992; Charrier and Thiébot, 1996; Evenden and Ryan, 1996; Richards et al., 1999; Cardinal et al., 2000; Isles et al., 2003; Helms et al., 2006; Bizot et al., 2007; Floresco et al., 2008).”

“The greater K or r, the more the individual is likely to choose the sooner option and therefore the more impulsive is the individual (Ho et al., 1999; Pine et al., 2009).”

“effect on the rate of slowing down engendered by decision– conflict (Frank et al., 2007; Pochon et al., 2008) to distinguish global from discrete influences on impulsivity.”

“, 2002) or no effect (Acheson and de Wit, 2008; Hamidovic et al., 2008) when enhancing dopamine function.”

“…et al., 2008), whereas others demonstrate the opposite, a dosedependent effect, or no effect (Logue et al., 1992; Charrier and Thiébot, 1996; Evenden and Ryan, 1996; Richards et al., 1999; Cardinal et al., 2000; Isles et al., 2003; Helms et al., 2006; Bizot et al., 2007; Floresco et al., 2008).”

“…et al., 2008), whereas others demonstrate the opposite, a dosedependent effect, or no effect (Logue et al., 1992; Charrier and Thiébot, 1996; Evenden and Ryan, 1996; Richards et al., 1999; Cardinal et al., 2000; Isles et al., 2003; Helms et al., 2006; Bizot et al., 2007; Floresco et al., 2008).”

“, 2008), whereas others demonstrate the opposite, a dosedependent effect, or no effect (Logue et al., 1992; Charrier and Thiébot, 1996; Evenden and Ryan, 1996; Richards et al., 1999; Cardinal et al., 2000; Isles et al., 2003; Helms et al., 2006; Bizot et al., 2007; Floresco et al., 2008).”

“Anatomical localization was performed by overlaying the t maps on a normalized structural image averaged across subjects and with reference to the anatomical atlas of Mai et al. (2003).”

“located) [90,91] and of the PFC, including OFC [92], and is sensitive to DA manipulations [93].”

“Interestingly, delay discounting seems to depend on the function of the ventral striatum (104) and of the PFC, including OFC (105) and its connections to the NAc (106), and is sensitive to DA manipulations (107).”

“sensitive to DA manipulations (Pine et al. 2010).”

“Hence, these data suggest that D2/D3 receptor signalling contributes to at least some aspects of impulsivity, consistent with a report that levodopa increased in delay discounting in healthy volunteers, an effect evident in every participant (Pine et al., 2010).”

“Also, hyperbolic discounting or impulsive choice for more immediate rewards in humans is related to dopamine activation (99).”

“It has been revealed (in ‘normal’ adults) that excess dopamine levels (by the administration of levodopa versus placebo) can selectively increase choice towards more immediate rewards, by influencing the rate of the discounting function, such that increases in delay appear to become intolerable (Pine et al., 2010).”

“Third, findings from nonhuman's delay discounting of drug reinforcers are heterogeneous; they do not suggest that drug administration, in and of itself, is sufficient to change this discounting (e.g., Harty et al. 2011; but see Pine et al. 2010).”

“A second framework involves dopamine’s more immediate effects on behavior (Frank et al., 2004; Niv et al., 2007; Robbins and Everitt, 2007; Salamone et al., 2007; Lex and Hauber, 2008; Palmiter, 2008; Sharot et al., 2009; Pine et al., 2010; Jocham et al., 2011; Guitart-Masip et al., 2012; Shiner et al., 2012; Norbury…”

“For the fMRI analysis,we estimated one single set of canonicalmodel parameters (αc+, αc−, αu+, αu−, τ) for all participants, similarly as in previous studies (e.g., Pine et al., 2010; Seymour et al., 2012; Voon et al., 2010).”

“Furthermore, administration of the DA precursor L-DOPA increases impulsive choice (Pine et al., 2010) in human volunteers, but does not affect SST or go/no-go performance (Overtoom et al., 2003; Hershey et al., 2004).”

“, 2010), converging evidence indicates that representations in the ventral striatum are modulated by whether outcomes are immediate versus delayed, and it has been shown repeatedly that the ventral striatum exhibits greater responses to the choice of sooner versus later outcomes (McClure et al., 2004; Kable and Glimch…”

“Dopaminergic medications can increase impulsivity in humans (Pine et al., 2010) and rodents (Logue et al.”

“Previous imaging studies have shown that amygdala activation was associated with individual differences in discounting, such that increased amygdala activity predicted more impulsive behavior (Hoffman et al., 2008; Pine et al., 2010).”

“Not surprisingly, delay discounting seems to depend on the function of the NAc (Gregorios-Pippas et al., 2009) and the PFC, including lateral OFC (Bjork et al., 2009), and is sensitive to DA manipulations (Pine et al., 2010).”

“Figure 2B demonstrates the effect of systemic dopamine agonists on discounting, adapted from Pine et al. (2010).”

“These deficits arise from dysregulation within frontal/basal ganglia circuitry that is often linked with dopamine dysfunction (Volkow et al., 2009; Pine et al., 2010).”

“Elevated dopamine function, in comparison, increases the ability of reward related cues to guide behavioral choices [65], diminishes the ability to differentiate between high and low value rewards [71], and induces steeper temporal discounting, a form of impulsivity defined by preference for immediately available smal…”

“In PD patients, administration of L-dopa correlated with greater attenuation of the dorsal striatum, insula, subgenual cingulate, and lateral orbitofrontal cortices for delayed relative to more immediate rewards, paralleling the behavioural result of increased impulsivity and temporal discounting relative to placebo […”

“…134; 1447–1463 P. A. MacDonald et al. D ow nloaded from https://academ ic.oup.com /brain/article-abstract/134/5/1447/286814 by U niversity of W estern O ntario user on 05 D ecem ber 2019 discounting) are combined to enact decisions (Pine et al., 2009, 2010; Ding and Gold, 2010; Cai et al., 2011).”

“The dopaminergic system is also suggested to play a role in impulsivity and delay of gratification [24],[39],[40], and some evidence suggests a potential role of”

“After correction for attenuation, random and scatter events, images were reconstructed using a Hanning filter of 2.0 mm into a volume of 47 slices, with a center-to-center distance of 3.125 mm and a pixel size of 2.02 mm×2.02 mm.”

“…incorporated into the construction of its ultimate value, suggesting a mechanism through which DA may influence human choice and temporal discounting, and traits such as impulsiveness that may account for several behavioral disorders associated with a hyperfunctioning DA system (Pine et al, 2010).”

“Alternatively, they may discount modestly, but exhibit steep diminishing marginal utility (Ho et al., 1999; Pine et al., 2009, 2010).”

“Second, human pharmacology studies suggest that DA modulates Now bias, albeit with inconsistent results (de Wit et al., 2002; Acheson and de Wit, 2008; Hamidovic et al., 2008; de Wit, 2009; Pine et al., 2010), although these studies have not accounted for intrinsic variations in DA signaling that could interact with p…”

“Both models rely on a softmax function for producing choice probabilities based on the values of the options including an inverse temperature (beta) parameter that determines the stochasticity of choice (see Pine et al., 2009, 2010 for details).”

“Evidence that some brain valuation sites do not respond to delayed rewards has been contradicted by research showing that all valuation sites respond to delayed prospects and discount them at the same rate (Kable and Glimcher 2007; Pine et al. 2010 ).”

“…broader range of functions (Berridge, 2007; Zhang et al , 2009) that are subject of much recent work (Palmiter, 2008; Rigoli et al , 2016a; Sharot, et al , 2009a; Pine et al , 2010; Jocham et al , 2011; Guitart-Masip et al , 2012; Shiner et al , 2012; Wunderlich et al , 2012; Norbury et al , 2013).”

“More recently, maximum likelihood estimation (MLE) has become a popular alternative to analyses based on indifference points (McClure et al. 2004; McClure et al. 2007; Pine et al. 2009; Pine et al. 2010).”

“Alternatively, he may discount modestly, but exhibits steep diminishing marginal utility such that the utility of the larger amount is only slightly larger than that of the smaller (Ho et al., 1999; Pine et al., 2009; Pine et al., 2010).”

“This choice function is frequently applied in computational modelling of reinforcement learning [60,61] and decisionmaking, including delay discounting [34,41,50,51,62,63] to convert value differences between decision options into choice probabilities.”

“A variety of evidence links delay-discounting to dopamine (DA) and DA-modulated frontostriatal circuits (Adriani et al. 2009; Altamirano et al. 2011; Boettiger et al. 2007; Doya 2008; Kobayashi and Schultz 2008; Lee et al. 2009; Mitchell et al. 2007; Paloyelis et al. 2010; Pine et al. 2010).”

“However, this seems unlikely since similar doses of L-DOPA given to healthy human volunteers induce behavioural effects [77– 79] and increase striatal DA synthesis [80].”

“…to THC may elicit persistent modifications in impulsivity is particularly interesting, in view of the proposed role of impulsivity in drug addiction and of the possible involvement of NAc shell DA in impulsivity (Basar et al. 2010; Besson et al. 2010; Pine et al. 2010; Sesia et al. 2010).”

“, 2009) and a correlation between striatal BOLD signals and the hyperbolic discount functions (Pine et al., 2009; Pine et al., 2010).”

“…of levodopa to healthy volunteers indeed increased impulsive choice of a small, immediate reward over a large, delayed reward by enhancing the diminishing influence of increasing delay on reward value (temporal discounting) and its corresponding neural re-presentation in the striatum [58].”

“Finally, a single study found effects opposite to our hypothesis -- increased delay discounting when augmenting DA using L-DOPA (150mg) (Pine et al., 2010).”

“Administration of the DA precursor L-dopa increases delayed reward discounting in healthy individuals [65].”

“2010 (95) 14 (6) Discounted utility 9 × Sasse et al.”

“However, the same dose of haloperidol did not affect performance on a slot machine game (Zack and Poulos, 2007), nor did a smaller dose (1.5 mg/kg, p.o.) affect rates of delay discounting in healthy adults (Pine et al., 2010).”

“Moreover, some data revealed a specific role for dopamine in controlling the relationship between the timing of future rewards and their subjective value [78].”

“Related to this, administration of dopamine antagonists to healthy participants increases the degree of temporal discounting, i.e., the relative value of delayed rewards decreases, leading to impulsive behavior (Pine et al., 2010).”

“Levodopa, a precursor to DA in healthy humans, increases impulsive choice (Pine et al. 2010) and increases delay aversion in patients with PD (Cools et al.”

“…recent study demonstrated that administration of L-Dopa to healthy volunteers engaged in an inter-temporal choice task enhanced VS BOLD activation to immediate rewards but attenuated VS BOLD activation to delayed rewards—an effect paralleled by an increase in delay discounting (Pine et al., 2010).”

“2010) and by pharmacological manipulations that heavily modulate subcortical DA in humans (Acheson and de Wit 2008; de Wit 2009; de Wit et al. 2002; Hamidovic et al. 2008; Pine et al. 2010).”

“[149] Furthermore, administration of the DA precursor L-3,4-dihydroxyphenylalanine (L-DOPA) can induce impulsive responding in healthy subjects[150] and in Parkinson’s disease patients.”

“However, other dopaminergic agents such as levodopa and pramipexole increase impulsivity on delay discounting and gambling tasks (24, 25).”

“Striatal dopamine is crucial for inhibitory control, updating working memory and decision-making processes (Cools et al., 2007; Hershey et al., 2004; Mehta et al., 2005; Pine et al., 2010).”

“Multiple studies have shown that areas of the brain with dopaminergic innervation are involved in temporal discounting preference (Kobayashi and Schultz, 2008; Pine et al., 2010).”

“We speculate that oxytocin might have reduced impulsiveness via interactions with the dopaminergic system as delay of gratification has been related to dopaminergic activity (Pine et al., 2010; Weber et al., 2016), whereby blocking dopaminergic neurotransmission increases delay of gratification similar to our current…”

“…that the effects of dopaminergic stimulation with pramipexole can significantly increase impulsive responding in non-lesioned rats (Madden et al., 2010; Johnson et al., 2011; Rokosik and Napier, 2012; Engeln et al., 2016), and that LD can increase impulsivity in healthy humans (Pine et al., 2010).”

“Often, the predictions of different accounts are even in apparent conflict with each other: for example, while some accounts claim that dopamine increases the tolerance for delay costs (8,9), others posit that dopamine enhances the preference for immediate rewards (10,11).”

“Commensurate with this hypothesis, systemic administration in humans of the dopamine precursor l-Dopa increases discount rates (Pine et al., 2010), although potentially countervailing evidence is that decreasing dopamine transmission in rats by administration of haloperidol (Denk et al., 2005) or…”

“…been linked to changes in striatal activity and to increases in impulsive-compulsive behavior and risk-taking choices (St Onge and Floresco 2009; Pine et al. 2010; O’Sullivan et al. 2011; Fineberg et al. 2010), suggesting more specific involvement of dopamine in appetitive reward value and in…”