Concurrent glucocorticoid and noradrenergic activity shifts instrumental behavior from goal-directed to habitual control

Pleasantness ratings dropped sharply after feeding for the devalued outcome but not for the valued outcome.

“pleasantness ratings decreased sharply after feeding for the devalued but not for the valued outcome (time ϫ value interaction: F (1,63) ϭ 30.76, p Ͻ 0.001, ␩ 2 ϭ 0.33).”

In neutral trials, participants did not choose the high-probability action more often than the low-probability action.

“In neutral trials, however, subjects did not chose the high-probability action more often than the low-probability action showing that they were indifferent as to whether they received the effectively neutral liquid or not.”

In the first extinction block, all participants preferred the valued high-probability action over its low-probability counterpart.

“All participants favored the valued high-probability action over its lowprobability counterpart in the first extinction block (*p Ͻ 0.05).”

Participants performed well on action-outcome knowledge tests, averaging 4.28 points in free recall and 8.04 points in cued recall.

“On average, they reached 4.28 points (SEM: 0.23; maximum score: 6 points) in the free recall test and 8.04 points (SEM: 0.13; maximum score: 9 points) in the cued recall test.”

The shift in instrumental responding occurred without changes in explicit task knowledge.

“this change in instrumental responding came without changes in explicit task knowledge”

• Also stated in: § Discussion · plain

  “we found here no effect of cortisol or noradrenergic activation on action-outcome knowledge”

An ANOVA comparing the last training block and first extinction block found a four-way value by time by cortisol by yohimbine interaction.

“This analysis revealed a four-way interaction effect (F (1,63) ϭ 4.25, p Ͻ 0.05, ␩ 2 ϭ 0.06).”

Hydrocortisone administration significantly raised salivary cortisol.

“the intake of hydrocortisone (i.e., cortisol) resulted in a significant increase in salivary cortisol.”

In the first 15-trial extinction block, participants given hydrocortisone alone or yohimbine alone preferred the valued but not the devalued high-probability action over the corresponding low-probability action.

“and preferred the valued but not the devalued high-probability action over the respective low-probability action in the first 15-trial block (valued: both t values Ͼ5.71, both p values Ͻ0.001; devalued: both t values Ͻ1.55, both p values Ͼ0.14).”

The combination of high cortisol and increased noradrenergic activity made participants' behavior insensitive to changes in goal value.

“Our results indicate that a combination of high cortisol concentrations and increased noradrenergic activity renders individuals' behavior insensitive to changes in goal value.”

• AssumptionRests on an assumption: Yohimbine-induced increases in blood pressure reflect increased noradrenergic activity.

In the first 15-trial extinction block, placebo participants favored the valued high-probability action over its low-probability counterpart.

“In the first 15-trial block, before they could know that the rewards were not presented any longer, they favored the val-ued high-probability action over its low-probability counterpart (binomial test, t (14) ϭ 8.05, p Ͻ 0.001).”

Yohimbine significantly raised systolic blood pressure.

“Yohimbine caused a significant activation of the autonomic nervous system as reflected in significant increases in systolic (main effect yohimbine and yohimbine ϫ time interaction: both F values Ͼ 13.66, both p values Ͻ0.001, both ␩ 2 Ͼ 0.18) (Fig. 1B)”

Placebo participants behaved goal-directed, preferring the high-probability action for the valued outcome over the devalued outcome across extinction testing.

“Participants that had received a placebo before learning behaved goal-directed.Consistent with their pleasantness ratings, they preferred the high-probability action associated with the valued outcome over the high-probability action associated with the devalued outcome across extinction testing (F (1,14) ϭ 4.88, p Ͻ 0.05, ␩ 2 ϭ 0.26).”

The four treatment groups had similar learning curves.

“Importantly, the four treatment groups did not differ in their learning curves.”

All participants decreased responding to the valued high-probability action as extinction testing proceeded.

“All participants decreased responding to the valued high-probability action as extinction testing proceeded (time ϫ cortisol ϫ yohimbine ANOVA; main effect time: F (4,252) ϭ 11.72, p Ͻ 0.001, ␩ 2 ϭ 0.16; for all other effects: p Ͼ 0.27) which suggests successful extinction learning.”

Reaction times were not affected by treatment or outcome devaluation.

“reaction times were not modulated by the treatment or the outcome devaluation”

• Also stated in: § No effect of cortisol, yohimbine, or outcome devaluation on reaction times · plain

  “there were no other main or interaction effects (all p values Ͼ0.10)”

In the first 15 extinction trials, participants in the cortisol plus yohimbine group favored both the valued and devalued high-probability actions over their low-probability counterparts.

“Furthermore, subjects in the cortϩyoh group favored both the valued and the devalued highprobability actions over their low-probability counterparts in the first 15 extinction test trials (both t (18) Ͼ 6.54, both p Ͻ 0.001).”

All participants except the cortisol plus yohimbine group markedly reduced responding to the devalued action from training to extinction testing.

“As shown in Figure 5 all participants decreased responding to the devalued action markedly from training to extinction testing (i.e., after outcome devaluation), except the participants in the cortϩyoh group.”

A mixed-design ANOVA found a significant value by cortisol by yohimbine interaction.

“a value (valued vs devalued) ϫ time (five 15 trial blocks) ϫ cortisol (yes vs no) ϫ yohimbine (yes vs no) mixeddesign ANOVA yielded a significant value ϫ cortisol ϫ yohimbine interaction effect (F (1,63) ϭ 3.88, p ϭ 0.05, ␩ 2 ϭ 0.06).”

Combined cortisol and yohimbine administration significantly altered instrumental behavior.

“The combined administration of cortisol and yohimbine, however, altered instrumental behavior significantly.”

Participants in the cortisol plus yohimbine group did not prefer the valued high-probability action over the devalued high-probability action.

“Participants in the cortϩyoh group did not prefer the valued highprobability action over the devalued high-probability action (F (1,19) ϭ 0.19, p ϭ 0.67, ␩ 2 ϭ 0.01).”

Across training, all groups increasingly chose the high-probability actions linked to chocolate milk and orange juice over the low-probability actions.

“Over training, participants in all groups increasingly preferred the highprobability actions associated with the rewards (chocolate milk and orange juice, respectively) over the referring low-probability actions.”

Participants given hydrocortisone alone or yohimbine alone behaved goal-directed, choosing valued high-probability actions more often than devalued high-probability actions across extinction.

“As in the plac group, participants in the placϩcort and placϩyoh groups performed goal-directed.They chose the valued high-probability actions more often than the devalued highprobability actions across extinction testing (both F values Ͼ8.03, both p values Ͻ0.02, both ␩ 2 Ͼ 0.33)”

Cortisol and yohimbine did not affect pleasantness ratings.

“There were no effects of cortisol or yohimbine on pleasantness ratings (all F values Ͻ1.69, all p values Ͼ0.20, all ␩ 2 Ͻ 0.03).”

For neutral trials, participants showed no preference for the high-probability action over the low-probability one.

“In neutral trials, subjects did not favor the high-probability action over the low-probability action.”

Prior to selective devaluation, both the valued and devalued rewards were rated as more pleasant than the common and neutral outcomes.

“Before the selective outcome devaluation, participants found the rewards (valued and devalued outcome) more pleasant than the common and neutral outcomes.”

Every participant learned the instrumental action linked to food rewards, increasingly favoring high-probability actions for chocolate milk and orange juice over the low-probability alternatives.

“All participants learned to choose the instrumental action associated with the food rewards; they increasingly preferred the high-probability actions associated with chocolate milk and orange juice, respectively, over their low-probability counterparts (*p Ͻ 0.05).”

Following devaluation, pleasantness ratings dropped significantly for the eaten food (devalued outcome) compared with the uneaten food (valued outcome).

“After devaluation, pleasantness ratings decreased significantly for the food eaten (devalued outcome) relative to the food not eaten (valued outcome).”

After selective devaluation, every group except the cortisol-plus-yohimbine group reduced responding to the devalued high-probability action.

“All but the participants of the Cortϩyoh group decreased responding to the devalued high-probability action after selective outcome devaluation (**p Ͻ 0.001).”

Stress and stress hormones reduce neuroplasticity within the prefrontal cortex.

“stress or stress hormones impair neuroplasticity in the prefrontal cortex (Diamond et al., 2007).”

• cites Diamond et al., 2007Not checked yet

  Not checked in the free run.

Compared with prefrontal cortex, dorsolateral striatum has fewer stress hormone receptors.

“the dorsolateral striatum expresses stress hormone receptors to a lesser extent (Patel et al., 2000)”

• cites Patel et al., 2000Not checked yet

  Not checked in the free run.

Cortisol and noradrenergic arousal act together to move instrumental behavior from goal-directed to habitual control.

“Thus, the present findings show that cortisol and noradrenergic arousal interact synergistically to shift instrumental behavior from goaldirected to habitual action.”

• Also stated in: § Discussion · plain

  “our findings show that GCs and noradrenaline act in concert to promote the switch from goal-directed to habitual performance.”

• AssumptionRests on an assumption: Yohimbine-induced increases in blood pressure reflect increased noradrenergic activity.

Yohimbine, an alpha2-adrenoceptor antagonist, raises brain noradrenaline levels.

“coadministering the ␣2-adrenoceptor antagonist yohimbine, which increases noradrenaline levels in the brain, with GCs in low-arousal conditions, reinstated GC effects on memory (Roozendaal et al., 2006a).”

• cites Roozendaal et al., 2006aNot checked yet

  Not checked in the free run.

For hippocampus-dependent spatial or declarative memory, glucocorticoid actions need simultaneous noradrenergic activation.

“it is by now well established that GC effects on hippocampus-dependent spatial or declarative memory necessitate co-occurring noradrenergic activation (Roozendaal et al., 2006a;de Quervain et al., 2007;Schwabe et al., 2009; for review, see Roozendaal et al., 2009).”

• cites Roozendaal et al., 2006aNot checked yet

  Not checked in the free run.

• cites de Quervain et al., 2007Not checked yet

  Not checked in the free run.

• cites Schwabe et al., 2009Not checked yet

  Not checked in the free run.

The decrease in responding to the valued high-probability action during extinction suggests successful extinction learning.

“which suggests successful extinction learning.”

Participants who received both cortisol and yohimbine reported no longer wanting the devalued outcome but still performed its associated action, indicating habitual responding.

“In sum, participants that had received both cortisol and yohimbine indicated that they did not want the devalued outcome any more but still performed the action associated with the devalued outcome, i.e., they responded habitually.”

• Also stated in: § Abstract · plain

  “The combined administration of hydrocortisone and yohimbine, however, rendered participants' behavior insensitive to changes in the value of the goal (i.e., habitual).”

Prefrontal cortex has a high density of receptors for stress hormones.

“The prefrontal cortex expresses stress hormone receptors at a high density (Patel et al., 2000)”

• cites Patel et al., 2000Not checked yet

  Not checked in the free run.

The findings show that concurrent glucocorticoid and noradrenergic activity shifts instrumental behavior from goal-directed to habitual control.

“These findings demonstrate that the concerted action of glucocorticoids and noradrenergic activity shifts instrumental behavior from goal-directed to habitual control.”

This study is the first to show that glucocorticoid actions on the interaction between prefrontal cortex and dorsolateral striatum memory systems also need concurrent noradrenergic activation.

“The present study shows for the first time that GC effects on the interplay of multiple memory systems (here the prefrontal cortex and the dorsolateral striatum) require also a co-occurrence of noradrenergic activation.”

• AssumptionRests on an assumption: Yohimbine-induced increases in blood pressure reflect increased noradrenergic activity.

Cortisol plus yohimbine may have reduced participants' capacity to combine cognitive and emotional information.

“which might indicate that the combined administration of cortisol and yohimbine impaired participants' ability to integrate cognitive and emotional information (Pessoa, 2008).”

• cites Pessoa, 2008Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: Yohimbine-induced increases in blood pressure reflect increased noradrenergic activity.

Newer findings indicate that glucocorticoids can strengthen memory processes dependent on the dorsolateral striatum.

“there is recent evidence showing that GCs enhance dorsolateral striatum-based memory processes (Medina et al., 2007;Quirarte et al., 2009).”

• cites Medina et al., 2007Not checked yet

  Not checked in the free run.

• cites Quirarte et al., 2009Not checked yet

  Not checked in the free run.

Giving yohimbine together with glucocorticoids under low arousal restored glucocorticoid effects on memory.

“coadministering the ␣2-adrenoceptor antagonist yohimbine, which increases noradrenaline levels in the brain, with GCs in low-arousal conditions, reinstated GC effects on memory (Roozendaal et al., 2006a).”

• cites Roozendaal et al., 2006aNot checked yet

  Not checked in the free run.

The learning data indicate successful instrumental learning.

“This indicates successful instrumental learning.”

In the devaluation paradigm used here and in most prior work, the influence of stress hormones on learning goal-directed versus habitual responses cannot be distinguished from their influence on performance.

“stress hormone effects on the acquisition of goal-directed and habitual responses cannot be separated from the expression of these responses in the devaluation paradigm that was used here and in most other studies on instrumental action”

Pharmacologically raising cortisol and noradrenergic activity can reproduce the stress-induced shift from goal-directed to habitual action reported earlier.

“Our data show that the previously reported switch from goaldirected to habitual action after stress (Schwabe and Wolf, 2009) can be mimicked by pharmacological elevations of cortisol and noradrenergic activity.”

• cites Schwabe and Wolf, 2009Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: Yohimbine-induced increases in blood pressure reflect increased noradrenergic activity.

In this study, action-outcome knowledge was generally very high, pointing to a ceiling effect.

“Action-outcome knowledge was overall very high in the present study which suggests a ceiling effect in performance.”

Hydrocortisone recipients showed significantly higher salivary cortisol before learning and before the extinction test.

“Participants who received hydrocortisone had significantly elevated salivary cortisol concentrations before learning and before the extinction test.”

When a stressor is given before extinction testing-after learning and after outcome devaluation-participants become insensitive to outcome devaluation.

“We showed recently that participants lose sensitivity to outcome devaluation when they are exposed to a stressor before extinction testing, i.e., after learning and after outcome devaluation (Schwabe and Wolf, 2010).”

• cites Schwabe and Wolf, 2010Not checked yet

  Not checked in the free run.

Lacking direct noradrenaline measurements, the study cannot exclude explanations that depend only on noradrenaline levels-for example, that hydrocortisone plus yohimbine raised noradrenaline above a threshold needed for the habit shift, a level yohimbine alone did not achieve.

“Without such data, however, we cannot fully rule out explanations that are based solely on noradrenaline concentrations, e.g., that the combined administration of hydrocortisone and yohimbine led to an above threshold concentration of noradrenaline which is necessary for the shift toward habit behavior and was not reached by yohimbine alone.”

Stress before learning seemed stronger and longer lasting than stress before retrieval.

“the effect of stress before learning appeared to be more pronounced and more long-lasting than the effect of stress before retrieval.”

• cites Schwabe and Wolf, 2009Not checked yet

  Not checked in the free run.

• cites Schwabe and Wolf, 2010Not checked yet

  Not checked in the free run.