Impaired probabilistic reversal learning in youths with mood and anxiety disorders

“All participants were evaluated by graduate-level clinicians with high inter-rater reliability (k>0.75) using the Kiddie-Schedule for Affective Disorders Present and Lifetime version (K-SADS-PL) administered to parents and children separately (Kaufman et al. 1997).”

“BD and SMD subjects completed the Young Mania Rating Scale (YMRS; Young et al. 1978).”

“Nevertheless, previous studies have demonstrated that ANX subjects, including those with trait anxiety, have deficits in paradigms requiring goal-directed stimulus attention and task switching, such as the Wisconsin Card Sorting Task (WCST; Eysenck et al. 2007).”

“…2002) and/or a Children’s Depression Rating Scale (CDRS) score >39 (Poznanski et al. 1985) ; (2) a Children’s Global Assessment Scale (CGAS) score<60 (Shaffer et al. 1983) ; and (3) persistent anxiety and/ or depression (consistently elevated PARS and/or CDRS) during 3 weeks of supportive…”

“…and adults after neurosurgical resection indicate that it is mediated by similar regions, including both the frontal cortex (dorsomedial, inferior and ventromedial) and the striatum (Blair et al. 2001 ; Cools et al. 2002, 2004 ; Clark et al. 2004 ; O’Doherty et al. 2004 ; Budhani et al. 2007).”

“2007) whereas the ventral striatum transforms concrete stimulus exemplar information into motor responses (Knutson et al. 2000 ; Cools et al. 2004).”

“…and adults after neurosurgical resection indicate that it is mediated by similar regions, including both the frontal cortex (dorsomedial, inferior and ventromedial) and the striatum (Blair et al. 2001 ; Cools et al. 2002, 2004 ; Clark et al. 2004 ; O’Doherty et al. 2004 ; Budhani et al. 2007).”

“For the BD group, children with irritable mania only, without elevated or expansive mood, were excluded (Geller et al. 1998 ; Leibenluft et al. 2003).”

“Moreover, reversal learning is mediated by the same frontostriatal circuit implicated in BD, MDD and ANX (Drevets, 2000 ; Thomas et al. 2001 ; Blumberg et al. 2002 ; Monk et al. 2006 ; Krain et al. 2008 ; Pfeifer et al. 2008).”

“PRR is of particular interest because the circuitry mediating its performance has been well studied in humans and non-human primates (Cools et al. 2002 ; Fellows & Farah, 2003 ; Clark et al. 2004 ; Izquierdo & Murray, 2004 ; Budhani et al. 2007 ; Bellebaum et al. 2008).”

“However, it is important to note that the WCST is thought to rely on attentionally based category shifting (Robbins et al. 1998).”

“That is, irritability in youths may be due in part to the frustration that occurs when they are unable to adapt to changing reward contingencies in the environment (Blair & Cipolotti, 2000 ; Blair, 2004).”

“Moreover, reversal learning is mediated by the same frontostriatal circuit implicated in BD, MDD and ANX (Drevets, 2000 ; Thomas et al. 2001 ; Blumberg et al. 2002 ; Monk et al. 2006 ; Krain et al. 2008 ; Pfeifer et al. 2008).”

“…study in healthy adults found that those taking the selective serotonin reuptake inhibitor citalopram had impaired probabilistic learning but unaltered response inhibition, whereas the opposite was true for those taking the noradrenaline reuptake inhibitor atomoxetine (Chamberlain et al. 2006).”

“Other regions that may play an accessory role include the parietal cortex, which is implicated in decision making in the presence of uncertainty (Paulus et al. 2001 ; Huettel, 2006 ; Chamberlain et al. 2008).”

“Although some studies show that SMD youths share some deficits with those with narrow-phenotype BD, including face emotion labeling (Guyer et al. 2007 ; Rich et al. 2008), others show differences between SMD and BD, including family history and longitudinal course (Brotman et al. 2006, 2007).”

“Moreover, reversal learning is mediated by the same frontostriatal circuit implicated in BD, MDD and ANX (Drevets, 2000 ; Thomas et al. 2001 ; Blumberg et al. 2002 ; Monk et al. 2006 ; Krain et al. 2008 ; Pfeifer et al. 2008).”

“PRR is of particular interest because the circuitry mediating its performance has been well studied in humans and non-human primates (Cools et al. 2002 ; Fellows & Farah, 2003 ; Clark et al. 2004 ; Izquierdo & Murray, 2004 ; Budhani et al. 2007 ; Bellebaum et al. 2008).”

“fMRI has not been used to study the circuitry mediating reversal learning in patients with MDD, but several regions of the PFC and the striatum have been implicated in the pathophysiology of depression (Milham et al. 2005 ; Rosenberg et al. 2005 ; Roberson-Nay et al. 2006 ; McClure et al. 2007).”

“Such grouping of ANX youths has been used by the Research Units on Pediatric Psychopharmacology (RUPP, 2001) and in other randomized controlled trials for pediatric anxiety (Birmaher et al. 2003).”

“By contrast, such adolescents did not have alterations in either the dorsomedial or inferior frontal cortex, suggesting an intact ability to alter motor responses (Finger et al. 2008).”

“Prepubertal and early adolescent bipolarity differentiate from ADHD by manic symptoms, grandiose delusions, ultra-rapid or ultradian cycling.”

“…Medicine (2010), 40, 1089–1100. f Cambridge University Press 2009 doi:10.1017/S0033291709991462 ORIGINAL ARTICLE (Blair et al. 2001 ; Clark et al. 2004 ; Cools et al. 2004 ; O’Doherty et al. 2004 ; Budhani et al. 2007) whereas the ventral striatum transforms…”

“However, emerging neuroimaging data using tasks other than reversal learning implicate several frontal regions, including dorsolateral, ventromedial and anterior cingulate cortices, along with the amygdala and striatum in the pathophysiology of pediatric BD (Blumberg et al. 2003 ; Rich et al. 2006 ; Dickstein et al. 2…”

“For the BD group, children with irritable mania only, without elevated or expansive mood, were excluded (Geller et al. 1998 ; Leibenluft et al. 2003).”

“…regions, including dorsolateral, ventromedial and anterior cingulate cortices, along with the amygdala and striatum in the pathophysiology of pediatric BD (Blumberg et al. 2003 ; Rich et al. 2006 ; Dickstein et al. 2007b ; Leibenluft et al. 2007 ; Nelson et al. 2007 ; Pavuluri et al. 2007, 2008).”

“Although some studies show that SMD youths share some deficits with those with narrow-phenotype BD, including face emotion labeling (Guyer et al. 2007 ; Rich et al. 2008), others show differences between SMD and BD, including family history and longitudinal course (Brotman et al. 2006, 2007).”

“2007) whereas the ventral striatum transforms concrete stimulus exemplar information into motor responses (Knutson et al. 2000 ; Cools et al. 2004).”

“Moreover, secondary analyses reveal that BD and SMD may show particular difficulty in using reinforcement expectancy information to guide their behavioral choices, as they are more likely to shift away from a correct response whether they have received either a reward or ‘spurious’ punishment (Blair, 2009).”

“However, emerging neuroimaging data using tasks other than reversal learning implicate several frontal regions, including dorsolateral, ventromedial and anterior cingulate cortices, along with the amygdala and striatum in the pathophysiology of pediatric BD (Blumberg et al. 2003 ; Rich et al. 2006 ; Dickstein et al. 2…”

“Other regions that may play an accessory role include the parietal cortex, which is implicated in decision making in the presence of uncertainty (Paulus et al. 2001 ; Huettel, 2006 ; Chamberlain et al. 2008).”

“Our study suggests that BD youths may have neural alterations in frontostriatal regions mediating reversal learning, especially those involved with the receipt of unexpected, spurious reward and punishment (Ramnani et al. 2004 ; Abler et al. 2008).”

“Specifically, we have shown that BD youths made more errors and required more trials and time to complete a simple reversal than controls (Dickstein et al. 2004).”

“fMRI has not been used to study the circuitry mediating reversal learning in patients with MDD, but several regions of the PFC and the striatum have been implicated in the pathophysiology of depression (Milham et al. 2005 ; Rosenberg et al. 2005 ; Roberson-Nay et al. 2006 ; McClure et al. 2007).”

“PRR is of particular interest because the circuitry mediating its performance has been well studied in humans and non-human primates (Cools et al. 2002 ; Fellows & Farah, 2003 ; Clark et al. 2004 ; Izquierdo & Murray, 2004 ; Budhani et al. 2007 ; Bellebaum et al. 2008).”

“Although some studies show that SMD youths share some deficits with those with narrow-phenotype BD, including face emotion labeling (Guyer et al. 2007 ; Rich et al. 2008), others show differences between SMD and BD, including family history and longitudinal course (Brotman et al. 2006, 2007).”

“We include youths with SMD because these patients suffer from attention deficit/hyperactivity disorder (ADHD)-like symptoms of hyperactivity plus severe irritability, and their diagnostic status vis-à-vis BD is unclear (Carlson, 1998 ; NIMH, 2001; Leibenluft et al. 2003).”

“fMRI has not been used to study the circuitry mediating reversal learning in patients with MDD, but several regions of the PFC and the striatum have been implicated in the pathophysiology of depression (Milham et al. 2005 ; Rosenberg et al. 2005 ; Roberson-Nay et al. 2006 ; McClure et al. 2007).”

“One previous study failed to find reversal learning deficits in MDD youths (n=30) versus controls (n=49) using the non-probabilistic ID/ ED task (Kyte et al. 2005).”

“However, both BD and SMD subjects were impaired on the compound reversal stage (i.e. reversals involving compound stimuli of white line drawings superimposed on purple shapes ; subjects must discern which of the two stimuli is linked to reward) (Dickstein et al. 2007a).”

“Moreover, reversal learning is mediated by the same frontostriatal circuit implicated in BD, MDD and ANX (Drevets, 2000 ; Thomas et al. 2001 ; Blumberg et al. 2002 ; Monk et al. 2006 ; Krain et al. 2008 ; Pfeifer et al. 2008).”

“Although some studies show that SMD youths share some deficits with those with narrow-phenotype BD, including face emotion labeling (Guyer et al. 2007 ; Rich et al. 2008), others show differences between SMD and BD, including family history and longitudinal course (Brotman et al. 2006, 2007).”

“For the reversal trials, we calculated win-stay and lose-stay percentages for both correct and incorrect responses following the receipt of reward or punishment (Budhani et al. 2006).”

“Moreover, reversal learning is mediated by the same frontostriatal circuit implicated in BD, MDD and ANX (Drevets, 2000 ; Thomas et al. 2001 ; Blumberg et al. 2002 ; Monk et al. 2006 ; Krain et al. 2008 ; Pfeifer et al. 2008).”

“fMRI has not been used to study the circuitry mediating reversal learning in patients with MDD, but several regions of the PFC and the striatum have been implicated in the pathophysiology of depression (Milham et al. 2005 ; Rosenberg et al. 2005 ; Roberson-Nay et al. 2006 ; McClure et al. 2007).”

“Our study suggests that BD youths may have neural alterations in frontostriatal regions mediating reversal learning, especially those involved with the receipt of unexpected, spurious reward and punishment (Ramnani et al. 2004 ; Abler et al. 2008).”

“…regions, including dorsolateral, ventromedial and anterior cingulate cortices, along with the amygdala and striatum in the pathophysiology of pediatric BD (Blumberg et al. 2003 ; Rich et al. 2006 ; Dickstein et al. 2007b ; Leibenluft et al. 2007 ; Nelson et al. 2007 ; Pavuluri et al. 2007, 2008).”

“PRR is of particular interest because the circuitry mediating its performance has been well studied in humans and non-human primates (Cools et al. 2002 ; Fellows & Farah, 2003 ; Clark et al. 2004 ; Izquierdo & Murray, 2004 ; Budhani et al. 2007 ; Bellebaum et al. 2008).”

“However, emerging neuroimaging data using tasks other than reversal learning implicate several frontal regions, including dorsolateral, ventromedial and anterior cingulate cortices, along with the amygdala and striatum in the pathophysiology of pediatric BD (Blumberg et al. 2003 ; Rich et al. 2006 ; Dickstein et al. 2…”

“In addition, compared to controls, fewer BD patients achieved this minimal competence (Gorrindo et al. 2005).”

“PRR is of particular interest because the circuitry mediating its performance has been well studied in humans and non-human primates (Cools et al. 2002 ; Fellows & Farah, 2003 ; Clark et al. 2004 ; Izquierdo & Murray, 2004 ; Budhani et al. 2007 ; Bellebaum et al. 2008).”

“However, both BD and SMD subjects were impaired on the compound reversal stage (i.e. reversals involving compound stimuli of white line drawings superimposed on purple shapes ; subjects must discern which of the two stimuli is linked to reward) (Dickstein et al. 2007a).”

“Other regions that may play an accessory role include the parietal cortex, which is implicated in decision making in the presence of uncertainty (Paulus et al. 2001 ; Huettel, 2006 ; Chamberlain et al. 2008).”

“Moreover, reversal learning is mediated by the same frontostriatal circuit implicated in BD, MDD and ANX (Drevets, 2000 ; Thomas et al. 2001 ; Blumberg et al. 2002 ; Monk et al. 2006 ; Krain et al. 2008 ; Pfeifer et al. 2008).”

“However, emerging neuroimaging data using tasks other than reversal learning implicate several frontal regions, including dorsolateral, ventromedial and anterior cingulate cortices, along with the amygdala and striatum in the pathophysiology of pediatric BD (Blumberg et al. 2003 ; Rich et al. 2006 ; Dickstein et al. 2…”

“By contrast, reversal learning tasks rely on the ability to update reinforcement values associated with different responses to objects (Mitchell et al. 2008 ; Xue et al. 2008).”

“By contrast, reversal learning tasks rely on the ability to update reinforcement values associated with different responses to objects (Mitchell et al. 2008 ; Xue et al. 2008).”

“…inconsistency between SMD’s performance on the ID/ED and PRR tasks may reflect the effect of co-morbid ADHD in increasing cognitive performance variability, suggesting the need for future comparisons to adequately powered samples of youths with primary ADHD (Ben Pazi et al. 2003 ; Doyle, 2006).”

“Indeed, recent data suggest that specific deficits in stimulus/reinforcement-based decision making are present in adults with generalized anxiety but not in those with social phobia (Devido et al. 2008).”

“From an affective neuroscience perspective, reversal learning taps into cognitive flexibility, defined as the ability to adapt one’s thinking and behavior in response to changing environmental conditions including rewards (Cools et al. 2004 ; Stemme et al. 2005).”

“We use unexpected context changes to elicit affective responses, motivated in part by the fact that affective disorders are associated with deficiencies in performing this kind of task (Adlerman et al., 2011; Dickstein et al., 2010).”

“Effects on probabilistic reward learning were not found among euthymic children with bipolar disorder (Dickstein et al. 2010, Gorrindo et al. 2005) or in unmedicated adults diagnosed with bipolar depression (Roiser et al. 2009).”

“[12] Dickstein DP, Finger EC, BrotmanMA, Rich BA, Pine DS, Blair JR, et al.”

“In different clinical populations, reversal learning tests applying 100% accurate feedback are not always used, but instead probabilistic reinforcement paradigms are used to investigate cognitive flexibility [21,26,67].”

“Dickstein et al. (2007, 2010) demonstrated that youth with BD exhibit clear deficits in reversal learning, but there were only a few small effects and marginal trends that differentiated youth with SMD from healthy controls.”

“However, elevated win-switch tendencies have been reported in youths with bipolar disorder, major depressive disorder, and anxiety disorder (Dickstein et al., 2010).”

“Dickstein and colleagues [20] used a reversal learning task in children and adolescents with a range of mood disorders.”

“As detailed elsewhere12,56, trials were categorized by phase (acquisition or reversal), accuracy (correct or incorrect, based on subject response), and reinforcement (reward or punishment), yielding nine events: (1) acquisition correct win (“acquisition correct”); (2) acquisition correct lose; (3) acquisition incorrec…”

“…perseveration; monoamine oxidase; tryptophan 57 hydroxylase 58 59 Introduction 60 Cognitive inflexibility is widely associated with depression (Dickstein et al, 2010), 61 schizophrenia (Leeson et al, 2009), obsessive-compulsive disorder (OCD) (Chamberlain et 62 al, 2006; Remijnse et al,…”

“Interestingly, behavioral flexibility is frequently impaired in individuals diagnosed with stress-induced mental illnesses such as anxiety or depression (Grant et al., 2001; Dickstein et al., 2010).”

“The paradigm we used was adapted from one on which we previously demonstrated behavioral impairments in pediatric populations, and whose mediating neural circuitry has been characterized in both adult and youth samples using fMRI (7,15-18,53).”

“, 2003) and bipolar disorder (Dickstein et al., 2009), to psychopathy (Mitchell et al.”

“These deficits are commonly observed in probabilistic reversal learning tests in which an initial response pattern associated with positive feedback in a highly probable manner is reversed such that an alternative response becomes associated with positive feedback in a highly probably manner (Dickstein et al., 2009; L…”

“, 2009), mood disorders (Dickstein et al., 2009) and psychopathic traits (Finger et al.”

“Not surprisingly, the impairment of this behavioral flexibility leads to neuropsychiatric conditions [8] such as obsessive-compulsive behaviors [9], substance abuse [10], post-traumatic stress [11] and mood disorders [9, 12].”

“…as well as indifference to negative consequences are frequently observed in manic patients (American Psychiatric Association, 2000) as well as in depressed and euthymic patients (Dickstein et al., 2010; Gorrindo et al., 2005; McKirdy et al., 2009; Pizzagalli et al., 2008; Roiser et al., 2009).”

“Impairments in reversal learning are associated with a wide range of behavioral abnormalities or psychiatric conditions characterized by impulsiveness and disinhibition [1,2,3,4,5], such as reactive aggression [6], psychopathy [7,8], Obsessive compulsive-disorder [9,10], severe conduct disorder [11], and bipolar disor…”

“For example, in contrast to studies of typically developing young people with SMD [13, 24], similarly defined severe mood problems in ASD youth were not associated with deficits in executive function or emotion recognition (except for identifying the facial expression of surprise) [27].”

“There is also evidence of impaired simplistic probabilistic reversal learning in youths at risk of BD (Dickstein et al., 2010), in addition to other learning deficits (Duek et al.”

“probabilistic reversal learning task (Dickstein et al. 2009; Hasler et al. 2009a, b; Eshel and Roiser 2010).”

“These include behavioral (Dickstein et al., 2010; Dombrovski et al., 2010), electrophysiological (Chase et al., 2011), brain imaging (Cools et al., 2002; Hampton & O'Doherty, 2007) and computational modeling (den Ouden et al., 2013; Glascher et al., 2009; Hampton et al., 2006) approaches.”

“Compared to HCs, adolescents with BD demonstrate increased errors on reversal stages of cognitive flexibility tasks, slower adaptation to accurate stimuli, and are less likely to meet learning criterion(Dickstein et al., 2010b; Dickstein et al., 2007; Gorrindo et al., 2005; Wegbreit et al., 2016).”

“The present sample is comparable to other studies of DD (e.g., Ahn et al., 2011) and of cognitive functioning in pediatric BD (e.g., Dickstein et al., 2010).”

“youths with either major depressive disorder, anxiety disorders, SMD, or TDCs in another study [14].”

“Given that psychiatric patients exhibit impaired PRLT performance (Dickstein et al., 2010; Dombrovski et al., 2010; Reddy et al., 2016; Waltz and Gold, 2007; Young and Geyer, 2015; Young and Markou, 2015), the data presented here suggest that incorporation of OT into existing treatment plans may improve deficient rewa…”

“…learning and reversal learning test was used because past studies demonstrated that individuals with bipolar disorder or other psychiatric disorders exhibit probabilistic reversal learning deficits (Waltz and Gold, 2007; Weiler et al., 2009; Dickstein et al., 2010; D’Cruz et al., 2013).”

“In pediatric BSD studies, there is evidence for impaired reversal learning on probabilistic reinforcement tasks (Dickstein et al., 2010; Gorrindo et al., 2005), as well as in adolescent BSD studies (Uro ˇ sevi ´ c, Halverson, Youngstrom, & Luciana, 2018), whereas others have found no such deficits…”

“Recent research reveals, however, that children with DMDD often fulfills the criteria for ADHD [11–13].”

“Highly irritable children also show disrupted reward learning, with difficulty detecting contingencies between responses and rewards, as well as impaired behavioral adaptation when contingencies change (e.g., Adleman et al., 2011; Dickstein et al., 2010).”

“have impaired cognitive flexibility in computerized reversal learning tasks [10].”

“PREL has been used previously in many studies investigating feedback-related decision making using behavioural (Dickstein et al., 2010; Dombrovski et al., 2010), brain imaging (Budhani, Marsh, Pine, & Blair, 2007; Chase, Swainson, Durham, Benham, & Cools, 2011; Cools, Clark, Owen, & Robbins, 2002;…”

“It is also debated whether anxiety affects reward learning ability: Dickstein et al. (2010) and Browning et al. (2015) did not find any association between anxiety level and overall reward learning rate.”

“These were at levels which were did not reach the threshold for being clinically significant for mild depression or anxiety and are unlikely to be responsible for the behavioral impairment observed as impaired reversal learning has only been observed in patients with severe depression or bipolar disorder (Dickstein et…”

“With respect to anxiety, a few studies have been conducted on its relationship to negative outcome and reward sensitivity on the PRL task [30, 65, 67], and one of these is particularly methodologically strong [67].”

“All of these have been found to function atypically in patients with MDD (e.g. Dickstein et al., 2010; Forbes et al., 2006; Hammar & Ardal, 2009; Kovacs, Joormann, & Gotlib, 2008) but may not be unique to the disorder.”

“Probabilistic reversal learning is impaired in youths with bipolar disorders compared to healthy control subjects [57].”

“…et al., 2011; Dickstein et al., 2010; Uran & Kilic, 2015), the children were not significantly (but only descriptively) worse than healthy controls (Dickstein et al., 2010; Uran & Kilic, 2015), although the associated effect size ( d = .74) could indicate a type II error (Dickstein et al., 2010).”

“Similar to adult literature, adolescent BPSDs are characterized by dysregulated reward learning (Dickstein et al. 2009) and increased reward sensitivity and approach-motivated behaviors being associated with elevated manic symptoms (Gruber et al. 2013).”

“Impairments in executive functions such as flexibility are found in many psychiatric disorders such as psychopathy [1,2]; conduct disorder (CD) [3]; obsessive-compulsive disorder [4,5]; schizophrenia [6,7], autism spectrum disorder [8] and bipolar disorder [9,10].”

“Hence, flexible adaptation to the change in learning contingencies is critical for healthy functioning and deficits in this adaptation are associated with psychopathology (Britton, Grillon, Lissek, Norcross, & Szuhany, 2013; Chun & Phelps, 1999; Dickstein et al., 2010).”

“…with disruptions in reward learning and reversal learning (the ability to flexibly update learned associations between a stimulus and response when the contingencies of the association change), particularly among adolescents (Dickstein et al., 2010; Morgan, Olino, McMakin, Ryan, & Forbes, 2013).”

“The significance of neurocognitive impairments in mood disorders among children and adolescents (herein, youths) has been well-documented (Dickstein et al., 2010; Frías, Palma, & Farriols, 2014; Pavuluri, West, Hill, Jindal, & Sweeney, 2009; Wagner, Müller, Helmreich, Huss, & Tadić, 2014).”

“The participant is required to adjust their choice behaviour in order to retain the high reward [35] (Figure 1c).”

“To investigate this, we analysed the raw power spectral density (PSD, in fT) during the inter-trial intervals (ITI), a period when participants were at rest, awaiting the next stimulus to make a new choice and perform the corresponding action sequence.”

“For example, variable deficits in RL differentiate bipolar and major depressive or broader mood dysregulation disorders (e.g., where impaired RL is observed in bipolar but not major depressive disorders relative to control participants; Adleman et al., 2011; Dickstein et al., 2010).”