CoChat

Catecholaminergic Regulation of Learning Rate in a Dynamic Environment

Open access
Jepma, Marieke, Murphy, Peter R., Nassar, Matthew R., Rangel-Gomez, Mauricio, Meeter, Martijn, Nieuwenhuis, Sander · 2016 · PLOS Computational Biology

The DOI matches its Crossref record; 98 of the 98 references it cites were found in a registry or an index.

Abstract

Adaptive behavior in a changing world requires flexibly adapting one's rate of learning to the rate of environmental change. Recent studies have examined the computational mechanisms by which various environmental factors determine the impact of new outcomes on existing beliefs (i.e., the 'learning rate'). However, the brain mechanisms, and in particular the neuromodulators, involved in this process are still largely unknown. The brain-wide neurophysiological effects of the catecholamines norepinephrine and dopamine on stimulus-evoked cortical responses suggest that the catecholamine systems are well positioned to regulate learning about environmental change, but more direct evidence for a role of this system is scant. Here, we report evidence from a study employing pharmacology, scalp electrophysiology and computational modeling (N = 32) that suggests an important role for catecholamines in learning rate regulation. We found that the P3 component of the EEG-an electrophysiological index of outcome-evoked phasic catecholamine release in the cortex-predicted learning rate, and formally mediated the effect of prediction-error magnitude on learning rate. P3 amplitude also mediated the effects of two computational variables-capturing the unexpectedness of an outcome and the uncertainty of a preexisting belief-on learning rate. Furthermore, a pharmacological manipulation of catecholamine activity affected learning rate following unanticipated task changes, in a way that depended on participants' baseline learning rate. Our findings provide converging evidence for a causal role of the human catecholamine systems in learning-rate regulation as a function of environmental change.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1005171

Cites

98 of 98 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 The predictive-inference task. A. The number outcomes in a low-noise (upper panel) and a high-noise (lower panel) block. The blue dots indicate the number outcomes on each trial; these were drawn from a normal distribution of which the mean (horizontal black line) changed at unsignaled moments (change points). Change points are indicated by the vertical lines, separately for obvious (straight lines) and non-obvious (dotted lines) change points. The SD of the number-generating distribution was constant within each block, but varied between blocks (either 10 or 25). B. Illustration of the task. On each trial, participants predict the next location on a number line, after which they see the actual location (outcome) and update their prediction for the next trial. The number-line locations correspond to the number outcomes shown in plot A.
Fig 1 The predictive-inference task. A. The number outcomes in a low-noise (upper panel) and a high-noise (lower panel) block. The blue dots indicate the number outcomes on each trial; these were drawn from a normal distribution of which the mean (horizontal black line) changed at unsignaled moments (change points). Change points are indicated by the vertical lines, separately for obvious (straight lines) and non-obvious (dotted lines) change points. The SD of the number-generating distribution was constant within each block, but varied between blocks (either 10 or 25). B. Illustration of the task. On each trial, participants predict the next location on a number line, after which they see the actual location (outcome) and update their prediction for the next trial. The number-line locations correspond to the number outcomes shown in plot A.CC BY · PubMed Central
Fig 2 Learning rate increases as a function of prediction error and following change points. A. Learning rate as a function of prediction error, SD of the generative distribution and treatment. Learning rate is plotted as a function of median absolute prediction error, averaged using running bins of 150 trials, pooled across participants. Solid and dashed lines indicate mean and SEM, respectively. Running bins with the same median prediction error were combined, and the average learning rate was computed across the resulting larger bin. Note that although we used running-average bins for plotting purposes, we used single-trial measures of prediction error and learning rate in our statistical analyses (see Methods). B. Learning rate as a function of trials after change point, SD of the generative distribution and treatment (left = placebo, right = atomoxetine).
Fig 2 Learning rate increases as a function of prediction error and following change points. A. Learning rate as a function of prediction error, SD of the generative distribution and treatment. Learning rate is plotted as a function of median absolute prediction error, averaged using running bins of 150 trials, pooled across participants. Solid and dashed lines indicate mean and SEM, respectively. Running bins with the same median prediction error were combined, and the average learning rate was computed across the resulting larger bin. Note that although we used running-average bins for plotting purposes, we used single-trial measures of prediction error and learning rate in our statistical analyses (see Methods). B. Learning rate as a function of trials after change point, SD of the generative distribution and treatment (left = placebo, right = atomoxetine).CC BY · PubMed Central
Fig 3 Relationships between prediction error, P3 amplitude and learning rate. A. Grand-average outcome-evoked P3 (average signal across a cluster of four centroparietal electrodes centered on locations CPz and Pz), in the placebo and atomoxetine session. B. P3 amplitude as a function of prediction error, SD of the generative distribution and treatment. To make these plots, single-trial P3 amplitudes were z-scored per participant and plotted as a function of median absolute prediction error, averaged using running bins of 150 trials, pooled across participants. Running bins with the same median prediction error were combined, and the average P3 amplitude was computed across the resulting larger bin. Solid and dashed lines indicate mean and SEM, respectively. C. P3 amplitude predicts learning rate. Z-scored learning rates are sorted into 20 bins according to z-scored single-trial P3 amplitude, pooled across participants. The lines show the linear fit to the unbinned single-trial data. D. Mediation model and results. *** p < .001, * p < .05.
Fig 3 Relationships between prediction error, P3 amplitude and learning rate. A. Grand-average outcome-evoked P3 (average signal across a cluster of four centroparietal electrodes centered on locations CPz and Pz), in the placebo and atomoxetine session. B. P3 amplitude as a function of prediction error, SD of the generative distribution and treatment. To make these plots, single-trial P3 amplitudes were z-scored per participant and plotted as a function of median absolute prediction error, averaged using running bins of 150 trials, pooled across participants. Running bins with the same median prediction error were combined, and the average P3 amplitude was computed across the resulting larger bin. Solid and dashed lines indicate mean and SEM, respectively. C. P3 amplitude predicts learning rate. Z-scored learning rates are sorted into 20 bins according to z-scored single-trial P3 amplitude, pooled across participants. The lines show the linear fit to the unbinned single-trial data. D. Mediation model and results. *** p < .001, * p < .05.CC BY · PubMed Central
Fig 4 Relationships between change-point probability and relative uncertainty, P3 amplitude and learning rate. A. Mediation models and results. *** p < .001, ** p < .01. B. Both change-point probability and relative uncertainty predict P3 amplitude. Change-point probability and relative uncertainty estimates are sorted into 20 bins according to z-scored single-trial P3 amplitude, pooled across participants. The lines show the linear fit to the unbinned single-trial data.
Fig 4 Relationships between change-point probability and relative uncertainty, P3 amplitude and learning rate. A. Mediation models and results. *** p < .001, ** p < .01. B. Both change-point probability and relative uncertainty predict P3 amplitude. Change-point probability and relative uncertainty estimates are sorted into 20 bins according to z-scored single-trial P3 amplitude, pooled across participants. The lines show the linear fit to the unbinned single-trial data.CC BY · PubMed Central
Fig 5 Baseline-dependent atomoxetine effects on learning rate. A. Each participant’s average learning rate in the placebo and atomoxetine session, separately for the trials with no change point, nonobvious change points, and obvious change points. The dots, connected with lines, represent individual participants’ learning rates in each session, and the bars indicate the group-mean learning rate. B. Across-subject relationship between learning rate in the placebo session and the atomoxetine effect on learning rate, separately for the trials with no change point, nonobvious change points, and obvious change points. C. Permutation distributions (100,000 permutations) of the correlation coefficients expected based on regression to the mean. Our observed correlations are indicated by red lines.
Fig 5 Baseline-dependent atomoxetine effects on learning rate. A. Each participant’s average learning rate in the placebo and atomoxetine session, separately for the trials with no change point, nonobvious change points, and obvious change points. The dots, connected with lines, represent individual participants’ learning rates in each session, and the bars indicate the group-mean learning rate. B. Across-subject relationship between learning rate in the placebo session and the atomoxetine effect on learning rate, separately for the trials with no change point, nonobvious change points, and obvious change points. C. Permutation distributions (100,000 permutations) of the correlation coefficients expected based on regression to the mean. Our observed correlations are indicated by red lines.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 5

  1. A Pfefferbaum, 1987, Skull thickness influences P3 amplitude, Psychopharmacol Bull, 23, 493
  2. E Donchin, 1988, Is the P300 component a manifestation of context updating?, Behavioral and Brain Sciences, 11, 357, 10.1017/S0140525X00058027 · DOI · Paper page
  3. MA Eckert, 2010, Looking forward with the locus coeruleus, Science
  4. AA Gibbs, 2014, Catechol-O-methyltransferase val158met genotype determines effect of reboxetine on emotional memory in healthy male volunteers, Journal of psychiatry & neuroscience: JPN, 39, E24, 10.1503/jpn.130131 · DOI
  5. M Hansenne, 2000, The p300 cognitive event-related potential. I. Theoretical and psychobiologic perspectives, Neurophysiol Clin, 30, 191

Cited by

See all 91

Metadata

Authors
Jepma, M., Murphy, P. R., Nassar, M. R., Rangel-Gomez, M., Meeter, M., & Nieuwenhuis, S.
Affiliations
  • Leiden University (NL)
  • University Medical Center Hamburg-Eppendorf (DE)
  • Brown University (US)
  • Vrije Universiteit Amsterdam (NL)
Corresponding author
Jepma, Marieke
Venue
PLOS Computational Biology
Published
2016-10-28
Volume
12
Issue
10
Pages
e1005171
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
OpenAlex
W2540211666
Cited by
91
References
98
Citations by year
2026: 8 · 2025: 7 · 2024: 11 · 2023: 8 · 2022: 12 · 2021: 10 · 2020: 14 · 2019: 13 · 2018: 7 · 2017: 1
FWCI
4.45
Type
journal article
Language
en
Open access
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Leiden Repository (Leiden University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • VU Research Portal (submittedVersion, other-oa)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
Funders
National Institute of Mental Health (F32MH102009)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Keywords
learning rate, catecholamines, norepinephrine, dopamine, P300, prediction error, environmental change, surprise, belief updating, adaptive learning, neuromodulation, computational models of learning
MeSH terms
Adaptation, Physiological, Adolescent, Adult, Brain Mapping, Catecholamines, Dopamine, Double-Blind Method, Female, Humans, Learning, Male, Neuronal Plasticity, Norepinephrine, Parietal Lobe, Ecosystem, Neurotransmitter Agents, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.