CoChat
Sign Up
Cited by 491Neurotransmitter Receptor Influence on BehaviorNeural and Behavioral Psychology StudiesAnxiety, Depression, Psychometrics, Treatment, Cognitive Processes

The DOI matches its Crossref record; 140 of the 140 references it cites were found in a registry or an index.

Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis

Open access
Huys, Quentin JM, Pizzagalli, Diego A, Bogdan, Ryan, Dayan, Peter·2013·Biology of Mood & Anxiety Disorders
DOI 10.1186/2045-5380-3-12PMC PMC3701611
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Depression is characterised partly by blunted reactions to reward. However, tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. We attempted to disentangle these factors with respect to anhedonia in the context of stress, Major Depressive Disorder (MDD), Bipolar Disorder (BPD) and a dopaminergic challenge. METHODS: Six behavioural datasets involving 392 experimental sessions were subjected to a model-based, Bayesian meta-analysis. Participants across all six studies performed a probabilistic reward task that used an asymmetric reinforcement schedule to assess reward learning. Healthy controls were tested under baseline conditions, stress or after receiving the dopamine D2 agonist pramipexole. In addition, participants with current or past MDD or BPD were evaluated. Reinforcement learning models isolated the contributions of variation in reward sensitivity and learning rate. RESULTS: MDD and anhedonia reduced reward sensitivity more than they affected the learning rate, while a low dose of the dopamine D2 agonist pramipexole showed the opposite pattern. Stress led to a pattern consistent with a mixed effect on reward sensitivity and learning rate. CONCLUSION: Reward-related learning reflected at least two partially separable contributions. The first related to phasic prediction error signalling, and was preferentially modulated by a low dose of the dopamine agonist pramipexole. The second related directly to reward sensitivity, and was preferentially reduced in MDD and anhedonia. Stress altered both components. Collectively, these findings highlight the contribution of model-based reinforcement learning meta-analysis for dissecting anhedonic behavior.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/2045-5380-3-12

Cites

140 of 140 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Task and typical behaviour.A: Task. Each trial had the following structure: 1) 500 ms presentation of a central fixation cross; 2) 500 ms presentation of face without a mouth; 3) 100 ms presentation of long (13 mm) or short (11.5 mm) mouth inside the face; 4) participants reported whether the mouth was long or short by key-press (‘Z’ or ‘/’ on US keyboard, counterbalanced); 5) Face without mouth remained on screen until participant response. Short and long stimuli were each presented 50 times per block in pseudorandom sequence avoiding more than three repetitions in a row. Adapted from [10]. B: Reward schedule. One response (counterbalanced across participants) had a higher reward expectation. Correct identification of that “rich” stimulus was more likely to be rewarded (75% probability) than correct identification of the other, “lean”, stimulus (30% probability). There was no punishment. If in doubt, choosing the more rewarded stimulus was beneficial. C: Surrogate simulated data showing prototypical response evolution. The dark bars show a hypothetical control group, developing a strong response bias towards the more rewarded response over the three blocks of 100 trials. The light bars show a prototypical treatment group with a reduced response bias. D-E: Surrogate simulated data generated from a simple reinforcement learning (‘Stimulus-action’) model. Both a reduction in reward sensitivity (D) and a reduction in learning rate (E) can roughly reproduce the pattern in the data (C). F: Percent correct responses for each of the 392 experimental sessions. Each black point represents one experimental session. Vertical bars demarcate datasets. Red horizontal line represents chance performance for each session. Four participants performed below chance (red). Sixty-three out of 392 experimental sessions were not fitted better than chance by model ‘Belief’ (binomial test; blue). Of these, 58 out of 63 were in the Stress dataset, in which performance was generally worst.
Figure 1 Task and typical behaviour.A: Task. Each trial had the following structure: 1) 500 ms presentation of a central fixation cross; 2) 500 ms presentation of face without a mouth; 3) 100 ms presentation of long (13 mm) or short (11.5 mm) mouth inside the face; 4) participants reported whether the mouth was long or short by key-press (‘Z’ or ‘/’ on US keyboard, counterbalanced); 5) Face without mouth remained on screen until participant response. Short and long stimuli were each presented 50 times per block in pseudorandom sequence avoiding more than three repetitions in a row. Adapted from [10]. B: Reward schedule. One response (counterbalanced across participants) had a higher reward expectation. Correct identification of that “rich” stimulus was more likely to be rewarded (75% probability) than correct identification of the other, “lean”, stimulus (30% probability). There was no punishment. If in doubt, choosing the more rewarded stimulus was beneficial. C: Surrogate simulated data showing prototypical response evolution. The dark bars show a hypothetical control group, developing a strong response bias towards the more rewarded response over the three blocks of 100 trials. The light bars show a prototypical treatment group with a reduced response bias. D-E: Surrogate simulated data generated from a simple reinforcement learning (‘Stimulus-action’) model. Both a reduction in reward sensitivity (D) and a reduction in learning rate (E) can roughly reproduce the pattern in the data (C). F: Percent correct responses for each of the 392 experimental sessions. Each black point represents one experimental session. Vertical bars demarcate datasets. Red horizontal line represents chance performance for each session. Four participants performed below chance (red). Sixty-three out of 392 experimental sessions were not fitted better than chance by model ‘Belief’ (binomial test; blue). Of these, 58 out of 63 were in the Stress dataset, in which performance was generally worst.CC BY · PubMed Central
Figure 2 Model performance.A: Model comparison. Group-level log Bayes factors δiBIC for each model relative to model ‘Belief’ across all datasets. A difference ≥10 in this measure is strong evidence for the model with the lower score. B: The parameter γ in the model largely captures the probability with which participants made a correct choice. Note that, by design of the task, this explicitly captures the effect of symmetric instructions and perceptual difficulties, rather than the asymmetric effect of rewards.
Figure 2 Model performance.A: Model comparison. Group-level log Bayes factors δiBIC for each model relative to model ‘Belief’ across all datasets. A difference ≥10 in this measure is strong evidence for the model with the lower score. B: The parameter γ in the model largely captures the probability with which participants made a correct choice. Note that, by design of the task, this explicitly captures the effect of symmetric instructions and perceptual difficulties, rather than the asymmetric effect of rewards.CC BY · PubMed Central
Figure 3 Correlates of anhedonia.A: Correlation coefficients for all pairwise correlations between questionnaire measures. All are highly significant (p<.01), except for the correlation between anhedonic depression and anxious anxiety, denoted by a red dot. B: Hierarchical weighted regression analysis across all datasets, involving all 255 participants with a full set of BDI, BDA and MASQ scores. The plots shows the linear coefficients between anhedonic depression (AD) score and the reward sensitivity and learning rate parameters ρ and ϵ. Each bars shows one linear coefficient; the red error bars indicate ± 1 standard error; and the green error bars indicate the 99.4% confidence interval (corresponding to a Bonferroni corrected level p=.05/8). AD is significantly and negatively correlated with the reward sensitivity ρ, but not significantly correlated with the learning rate ϵ. C: Scatter plot of anhedonic depression against reward sensitivity. Size of dots scale with weight (inference precision). D: Scatter plot of reward sensitivity vs. learning rate. E: Significance of correlations across parameter estimates from 70 surrogate datasets. There is a consistent and stably significant correlation between AD and reward sensitivity ρ, but not between AD and learning rate ϵ.
Figure 3 Correlates of anhedonia.A: Correlation coefficients for all pairwise correlations between questionnaire measures. All are highly significant (p<.01), except for the correlation between anhedonic depression and anxious anxiety, denoted by a red dot. B: Hierarchical weighted regression analysis across all datasets, involving all 255 participants with a full set of BDI, BDA and MASQ scores. The plots shows the linear coefficients between anhedonic depression (AD) score and the reward sensitivity and learning rate parameters ρ and ϵ. Each bars shows one linear coefficient; the red error bars indicate ± 1 standard error; and the green error bars indicate the 99.4% confidence interval (corresponding to a Bonferroni corrected level p=.05/8). AD is significantly and negatively correlated with the reward sensitivity ρ, but not significantly correlated with the learning rate ϵ. C: Scatter plot of anhedonic depression against reward sensitivity. Size of dots scale with weight (inference precision). D: Scatter plot of reward sensitivity vs. learning rate. E: Significance of correlations across parameter estimates from 70 surrogate datasets. There is a consistent and stably significant correlation between AD and reward sensitivity ρ, but not between AD and learning rate ϵ.CC BY · PubMed Central
Figure 4 Comparing models incorporating hypotheses about group differences in terms of reward sensitivityρ or learning rateϵ. Each bar of the large panel shows the Bayes factor comparing ρ to ϵ. Green bars indicate very strong evidence for model ρ (Bayes factor ≥20), yellow bars weak evidence (Bayes factor 3−10) and cyan bars insufficient evidence (Bayes factor <3). MDD and high scores of anhedonic depression (AD) result in a reduction in ρ. Pramipexole instead reduces the learning rate. Stress has no differential effect on reward sensitivity compared with learning rate. Participants with a past history of depression, or euthymic individuals with bipolar disorder, showed weak evidence for a reduction in ρ. The insets at the top show the inferred priors over the relevant parameters of the winning model for each group.
Figure 4 Comparing models incorporating hypotheses about group differences in terms of reward sensitivityρ or learning rateϵ. Each bar of the large panel shows the Bayes factor comparing ρ to ϵ. Green bars indicate very strong evidence for model ρ (Bayes factor ≥20), yellow bars weak evidence (Bayes factor 3−10) and cyan bars insufficient evidence (Bayes factor <3). MDD and high scores of anhedonic depression (AD) result in a reduction in ρ. Pramipexole instead reduces the learning rate. Stress has no differential effect on reward sensitivity compared with learning rate. Participants with a past history of depression, or euthymic individuals with bipolar disorder, showed weak evidence for a reduction in ρ. The insets at the top show the inferred priors over the relevant parameters of the winning model for each group.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 3

  1. Beck A, Steer R, Brown G: Manual for the Beck Depression Inventory-II. 1996, San Antonio: Psychological Corporation
  2. Daw N: Decision Making, Affect, and Learning: Attention and Performance XXIII. Edited by: Delgado MR, Phelps EA, Robbins TW. 2009, OUP
  3. Dutra S, Brooks N, Lempert K, Guardado A, Goetz E, Pizzagalli D: Reward responsiveness in a remitted depressed sample: Effects of gender and trait negative affect. 23rd Annual Meeting of the Society for Research in Psychopathology. 2009, Minneapolis:

Cited by

Sorted by citations

See all 491

Related

See all

Metadata

Authors
Huys, Q. J., Pizzagalli, D. A., Bogdan, R., & Dayan, P.
Affiliations
  • Gatsby Computational Neuroscience Unit (GB)
  • Harvard University (US)
  • Duke University (US)
Corresponding author
Huys, Quentin JM
ORCID
Venue
Biology of Mood & Anxiety Disorders
Publisher
Springer Science and Business Media LLC
Published
2013-06-19
Volume
3
Issue
1
Pages
12-12
ISSN
2045-5380 (ISSN-L)
DOI
10.1186/2045-5380-3-12
PMC
PMC3701611
OpenAlex
W2157813419
Cited by
480
References
139
Citations by year
2026: 27 · 2025: 42 · 2024: 35 · 2023: 54 · 2022: 50 · 2021: 57 · 2020: 78 · 2019: 20 · 2018: 35 · 2017: 38 · 2016: 14 · 2015: 19 · 2014: 9 · 2013: 2
FWCI
7.20
Type
journal article
Language
en
Open access
Licence
Locations
  • Biology of Mood & Anxiety Disorders (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
  • Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
  • UCL Discovery (University College London) (submittedVersion)
  • MPG.PuRe (Max Planck Society) (submittedVersion)
  • Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
  • ETH Zürich Research Collection
  • Zurich Open Repository and Archive (University of Zurich)
Funders
  • AstraZeneca
  • Gatsby Charitable Foundation
  • Deutsche Forschungsgemeinschaft
  • Servier
  • National Institute of Mental Health (R01MH101521, R01MH095809, MH095809, R01MH068376, R21MH078979, MH068376)
  • National Center for Complementary and Integrative Health (R21AT002974)
Topics
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
Keywords
anhedonia, reinforcement learning, reward sensitivity, prediction error, dopamine, pramipexole, major depressive disorder, bipolar disorder, stress, learning rate, probabilistic reward task, model-based reinforcement learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review