CoChat
Sign Up
Cited by 164Neurotransmitter Receptor Influence on BehaviorMemory and Neural MechanismsNeural and Behavioral Psychology Studies

Serotonin neurons modulate learning rate through uncertainty

Open access
Grossman, Cooper D., Bari, Bilal A., Cohen, Jeremiah Y.·2022·Current Biology
DOI 10.1016/j.cub.2021.12.006PMC PMC8825708

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change are difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.

AI snapshot

Written from the paper’s full text.

TL;DR

Mice performing dynamic foraging and Pavlovian tasks showed choice and neural signatures consistent with meta-learning, where learning rate is adjusted by expected and unexpected uncertainty. Dorsal raphe serotonin neuron activity tracked these uncertainty variables, and chemogenetic inhibition of these neurons altered learning at reward-probability transitions in a way predicted by removing meta-learning from a reinforcement-learning model.

Key findings

  • In dynamic foraging, mice completed 278 ± 103 trials per session, chose the higher- or equal-probability spout at 0.66 ± 0.038, and earned 0.55 ± 0.027 rewards per trial. A meta-learning model fit behavior better than a static-learning model and reproduced slower adaptation after medium-to-low versus high-to-low reward-probability transitions.
  • Among 66 dorsal raphe serotonin neurons recorded from 4 mice, 33 (50%) correlated with expected uncertainty during inter-trial intervals; 26 correlated with expected uncertainty during cue/choice periods, and 17 correlated with unexpected uncertainty at outcome, mostly positively (15 of 17).
  • In a dynamic Pavlovian task, 25 of 41 serotonin neurons (61%) from 5 mice correlated with expected uncertainty, and 11 of 41 correlated with unexpected uncertainty at outcome.
  • Chemogenetic inhibition with hM4Di in serotonin neurons (6 mice, with 1 excluded; 6 mCherry controls) produced transition-learning changes matching a simulated lesion of meta-learning, while control mice showed no agonist 21 effect.

Methods

  • Head-restrained mice performed a two-spout dynamic foraging task with probabilistic water rewards drawn from {0.1, 0.5, 0.9} or {0.1, 0.4, 0.7}, changing in 20–35-trial blocks; a separate Pavlovian task used an odor cue predicting probabilistic reward after a delay.
  • Dorsal raphe serotonin neurons were recorded extracellularly in Sert-Cre mice and identified by optogenetic tagging; chemogenetic inhibition used hM4Di-mCherry and agonist 21 versus vehicle.
  • Behavior was modeled with Q-learning variants, including a static-learning model and a meta-learning model where expected uncertainty is a moving average of unsigned prediction errors and unexpected uncertainty updates the negative-RPE learning rate; models were fit hierarchically in Stan, compared by cross-validation, and analyzed with logistic regressions and linear mixed-effects models.

Limitations

  • The paper states that dynamic Pavlovian behavior did not distinguish meta-learning from static learning, that only the negative-RPE learning rate was subject to meta-learning, and that task asymmetries and binary outcomes may explain this asymmetry; it also notes possible contributions from multiple learning systems.
  • The paper acknowledges that autocorrelation controls may provide only a lower bound on true correlation rates.
  • Evident from the text: neural evidence is largely correlational, causal evidence comes from indirect chemogenetic inhibition, and electrophysiological samples are small (66 neurons from 4 mice; 41 neurons from 5 mice), which may limit generalizability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.cub.2021.12.006

Cites

Statistics

Nothing to recompute.

Claims the paper makes

0 of 94 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    When entering a new block, expected uncertainty became lower in the high block relative to the medium block within approximately 5 trials (4.80 ± 1.34).

    “We found that when entering a new block (from the animals’ perspective), expected uncertainty became lower in the high block relative to the medium block within approximately 5 trials (4.80 ± 1.34).”
    Quote in the paperSubstantial claim
  • Findingown data

    Recent rewards sped up response times, with a time constant of 1.88 ± 0.18 trials.

    “this model demonstrated a large effect of recent rewards on speeding up response times (Figure S1E; time constant, 1.88 ±0.18 trials, 95% CI).”
    Quote in the paperNot a substantial claim
  • Findingown data

    When reward probabilities flipped from high-low to low-high, mice rapidly shifted their choices to the new higher-probability alternative.

    “When the probability assignments flipped from high and low to low and high (for example, from 0.9 on the left and 0.1 on the right to 0.1 on the left and 0.9 on the right), mice rapidly shifted their choices to the new higher-probability alternative.”
    Quote in the paperSubstantial claim
  • Findingown data

    No clear behavioral evidence of variable learning rates was found in the Pavlovian task.

    “interestingly, we found no clear behavioral evidence of variable learning rates (Figures S5C–S5E).”
    Quote in the paperNot a substantial claim
  • Findingown data

    The updating rate for expected uncertainty from the firing rate model covaried with the same parameter from the choice model across sessions (R2 = 0.228, p = 0.003).

    “We found that the updating rate for expected uncertainty from the firing rate model covaried with the same parameter from the choice model across sessions (Figures 4G and 4H; R2 = 0.228, p = 0.003, linear regression).”
    Quote in the paperSubstantial claim
  • May combine two claims. 30
  • Classified as probably not a substantial claim. 23

  • May combine two claims.

    Findingown data

    A static learning rate model fit overall behavior well but could not capture a specific feature of behavior around transitions in reward probabilities.

    “This model fit overall behavior well,43,47 but was unable to capture a specific feature of behavior around transitions in reward probabilities (Figures 2D and 2F).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [43] Bari, 2019, Stable representations of decision variables for flexible behavior, Neuron source

      Not checked in the free run.

    • cites [47] Hattori, 2019, Area-specificity and plasticity of history-dependent value coding during learning, Cell source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Mice harvested rewards at a rate of 0.55 ± 0.027 rewards per trial.

    “and harvested rewards (reward rate, 0.55 ± 0.027 rewards trial−1)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The model was capable of explaining behavior and accurately estimating reward probabilities in the Pavlovian task.

    “While the model was capable of explaining behavior and accurately estimating reward probabilities (Figure S5B)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the Pavlovian task, serotonin neuron activity correlated with expected uncertainty at similar rates to the foraging task (61%, 25 of 41 neurons from 5 mice), and these correlations were mostly negative (20 of 25).

    “recordings of dorsal raphe serotonin neurons from mice behaving in this task revealed that the activity of these neurons correlated with expected uncertainty at similar rates to those recorded in the dynamic foraging task (Figures 6D–6G and S5F–S5H; 61%, 25 of 41 neurons from 5 mice) and were mostly negatively correlated (20 of 25).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Logistic regressions showed that mice used recent outcomes to drive behavior, with time constants of 1.35 ± 0.24 trials for rewards and 1.03 ± 0.14 trials for no rewards.

    “These logistic regressions revealed that mice used experience of recent outcomes to drive behavior (Figure 1C; time constants, 1.35 ± 0.24 trials for rewards, 1.03 ± 0.14 trials for no rewards, 95% confidence interval [CI]).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Serotonin neuron inhibition did not slow response times, change how outcomes drove response times, or cause mice to lick during inter-trial intervals.

    “Serotonin neuron inhibition did not slow response times (Figure S6B), change how outcomes drove response times (Figure S6C), nor cause mice to lick during inter-trial intervals.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    The block length experienced by the animal was often shorter (8.65 ± 2.83 trials) than the prescribed 20-35 trials because probabilities changed independently at each spout and animals switched choices.

    “While block lengths were prescribed to be 20–35 trials long, the block length experienced by the animal was often shorter (8.65 ± 2.83) due to the probabilities changing independently at each spout and the animals switching choices (which begins a new experienced block).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Firing rates correlated with unexpected uncertainty primarily during the outcome (17 of 66), and these correlations were mostly positive (15 of 17).

    “By contrast, firing rates correlated with unexpected uncertainty (v(t)) primarily during the outcome (17 of 66; Figures 5A–5D). These correlations were mostly positive (15 of 17).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The activity of most serotonin neurons distinguished rewards from no rewards during the outcome period, and many maintained this representation during the inter-trial interval.

    “The activity of most serotonin neurons distinguished rewards from no rewards during the outcome period and many neurons maintained this representation during the inter-trial interval (Figures 3E and 3G).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When reward probabilities changed, the deviation from expected uncertainty was greater for high-to-low changes (t626 = −13.0, p < 10−33), resulting in faster learning rates (t626 = −13.7, p < 10−36).

    “When the reward probabilities changed, the deviation from expected uncertainty was greater when high changed to low (t626 = −13.0, p < 10−33, two-sample t test), resulting in faster learning rates (t626 = − 13.7, p < 10−36, two-sample t test).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Not all forms of meta-learning mimicked mouse behavior; a previously proposed model for serotonin function, a Pearce-Hall model, and a model without expected value estimates were unsuccessful.

    “Interestingly, not all forms of meta-learning were capable of mimicking mouse behavior. We were unable to reproduce the observed behavior using a model previously proposed to modulate learning rates and explain serotonin neuron function (Figure S3).24,49 A Pearce-Hall model,50 which modulates learning as a function of RPE magnitude in a different way, was also unsuccessful, as was a model without”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [24] Daw, 2002, Opponent interactions between serotonin and dopamine, Neural Netw. source

      Not checked in the free run.

    • cites [49] Wittmann, 2020, Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys, Nat. Commun. source

      Not checked in the free run.

    • cites [50] Pearce, 1980, A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli, Psychol. Rev. source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Mice mostly chose the higher- or equal-probability spout, with a correct rate of 0.66 ± 0.038.

    “Mice mostly chose the higher- or equal-probability spout (Figure S1D; correct rate, 0.66 ± 0.038)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Mouse performance was better than a random agent but worse than an optimized one.

    “Mouse performance was better than a random agent, but worse than an optimized one (Figure S1D).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the Pavlovian task, serotonin neuron firing rates correlated with expected uncertainty throughout its update interval (27 of 41 during cue and delay, 25 of 41 after outcome) and with unexpected uncertainty at the time of the outcome (11 of 41).

    “Serotonin neuron firing rates also correlated with expected uncertainty throughout its update interval (27 of 41 during cue and delay, 25 of 41 after outcome; Figures 6E and 6I) and with unexpected uncertainty at the time of the outcome (11 of 41; Figure 6I).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The meta-learning RL model as formulated applies meta-learning only to the learning rate for negative reward prediction errors; this was an empirical result.

    “In the meta-learning RL model as we have formulated it, only the negative RPE learning rate is subject to meta-learning. This is an empirical finding”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Brief changes in serotonin neuron firing rates at the time of outcome correlated with unexpected uncertainty, consistent with previous work showing serotonin neuron activity correlated with surprise when cue-outcome relationships were violated.

    “Our observation that brief changes in serotonin neuron firing rates at the time of outcome correlated with unexpected uncertainty is also consistent with previous work showing that serotonin neuron activity correlated with “surprise” when cue-outcome relationships were violated.33”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] Matias, 2017, Activity patterns of serotonin neurons underlying cognitive flexibility, eLife source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The simulated firing rates had autocorrelation functions and distributions similar to those of the real neurons.

    “We found that the autocorrelation functions and the distributions of simulated firing rates were similar to those of the real neurons.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The updating rate of expected uncertainty allows calculation of expected uncertainty and detection of probability changes on timescales relevant to the task and behavior.

    “Thus, the updating rate of expected uncertainty allows for the calculation of expected uncertainty and detection of probability changes on timescales relevant to the task and behavior.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Most individual neurons correlated with only one form of uncertainty, suggesting that the computation is performed across the population.

    “While some individual neurons had significant correlations for both forms of uncertainty (2 neurons, CS-ε and outcome-v; 3 neurons, inter-trial-interval-ε and outcome-v; and 4 neurons with significant correlations with those variables during all 3 epochs), most only correlated with one (5 of 66 neurons CS-ε 11 inter-trial-interval-ε, and 8 outcome-v), suggesting that the computation is performed”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Unreliability of a slower learning system around transitions may enhance contributions from a faster learning system.

    “Unreliability of a slower learning system around transitions may enhance contributions from a faster learning system, for example.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    In the model, the negative RPE learning rate is a function of recent deviations from expected uncertainty and may be most related to a subjective estimate of environmental volatility.

    “In our model, the negative RPE learning rate is a function of recent deviations from expected uncertainty and thus may be most related to a subjective estimate of environmental volatility.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    The uncertainty representations observed may be related to the uncertainty or reliability of one of those learning systems, consistent with their generalization across actions.

    “From this perspective, the uncertainty representations we observed may be related to the uncertainty or reliability of one of those learning systems, consistent with their generalization across actions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Dorsal raphe serotonin neurons receive input from the locus coeruleus.

    “Dorsal raphe serotonin neurons also receive input from locus coeruleus.66”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [66] Ogawa, 2014, Organization of monosynaptic inputs to the serotonin and dopamine neuromodulatory systems, Cell Rep. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Because no trials had P(R)=0.1 for both spouts and mice often chose the P(R)=0.5 spout, rewards provided more information about which spout was likely good enough (P(R)=0.9 or 0.5).

    “Asymmetries in the task structure (the absence of trials in which P(R) = 0.1 for both spouts) and mouse preference (mice regularly exploited the P(R) = 0.5 spout) also result in rewards carrying more information about which spout is likely “good enough” (P(R) = 0.9 or P(R) = 0.5).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Both norepinephrine neurons in the locus coeruleus and acetylcholine neurons in the basal forebrain receive functional input from dorsal raphe serotonin neurons.

    “It should be noted that both norepinephrine neurons in the locus coeruleus64 and acetylcholine neurons in the basal forebrain65 receive functional input from dorsal raphe serotonin neurons.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [64] Szabo, 2001, Functional and pharmacological characterization of the modulatory role of serotonin on the firing activity of locus coeruleus norepinephrine neurons, Brain Res. source

      Not checked in the free run.

    • cites [65] Bengtson, 2004, Opposing electrophysiological actions of 5-HT on noncholinergic and cholinergic neurons in the rat ventral pallidum in vitro, J. Neurophysiol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation

    The results may also be consistent with models that combine contributions of different learning systems.

    “Our model proposes one learning system with variable learning rates, but these results may also be consistent with models that combine contributions of different learning systems.73–75”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [73] Daw, 2005, Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control, Nat. Neurosci. source

      Not checked in the free run.

    • cites [74] Lee, 2014, Neural computations underlying arbitration between model-based and model-free learning, Neuron source

      Not checked in the free run.

    • cites [75] Kim, 2019, Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning, Nat. Commun. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The activity of serotonin neurons changes on behaviorally relevant timescales, both fast (hundreds of milliseconds) and slow (tens of seconds).

    “The activity of these neurons changes on behaviorally relevant timescales—both fast (hundreds of milliseconds) and slow (tens of seconds).30–33”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [30] Liu, 2014, Dorsal raphe neurons signal reward through 5-HT and glutamate, Neuron source

      Not checked in the free run.

    • cites [31] Cohen, 2015, Serotonergic neurons signal reward and punishment on multiple timescales, eLife source

      Not checked in the free run.

    • cites [32] Li, 2016, Serotonin neurons in the dorsal raphe nucleus encode reward signals, Nat. Commun. source

      Not checked in the free run.

    • cites [33] Matias, 2017, Activity patterns of serotonin neurons underlying cognitive flexibility, eLife source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Serotonin receptor activation can induce short-term changes in excitability and long-lasting synaptic plasticity.

    “Serotonin receptor activation can induce short-term changes in excitability34,35 as well as long-lasting synaptic plasticity.36”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [34] Andrade, 2011, Serotonergic regulation of neuronal excitability in the prefrontal cortex, Neuropharmacology source

      Not checked in the free run.

    • cites [35] Celada, 2013, Serotonin modulation of cortical neurons and networks, Front. Integr. Neurosci. source

      Not checked in the free run.

    • cites [36] Lesch, 2012, Serotonin in the modulation of neural plasticity and networks: implications for neurodevelopmental disorders, Neuron source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Several theories propose that neuromodulatory systems enable meta-learning.

    “Several theories propose that neuromodulatory systems enable meta-learning.5,8,24”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [5] Doya, 2002, Metalearning and neuromodulation, Neural Netw. source

      Not checked in the free run.

    • cites [8] Yu, 2005, Uncertainty, neuromodulation, and attention, Neuron source

      Not checked in the free run.

    • cites [24] Daw, 2002, Opponent interactions between serotonin and dopamine, Neural Netw. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The paper's notion of expected uncertainty is related to reward variance, risk, or outcome uncertainty, but with respect to an entire behavioral policy rather than a specific action.

    “Our notion of expected uncertainty is also related to reward variance, risk, or outcome uncertainty,10,11,67–69 but with respect to an entire behavioral policy as opposed to a specific action.”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [10] Preuschoff, 2006, Neural differentiation of expected reward and risk in human subcortical structures, Neuron source

      Not checked in the free run.

    • cites [11] Preuschoff, 2007, Adding prediction risk to the theory of reward learning, Ann. N Y Acad. Sci. source

      Not checked in the free run.

    • cites [67] Bach, 2012, Knowing how much you don’t know: a neural organization of uncertainty estimates, Nat. Rev. Neurosci. source

      Not checked in the free run.

    • cites [68] Balasubramani, 2015, A network model of basal ganglia for understanding the roles of dopamine and serotonin in reward-punishment-risk based decision making, Front. Comput. Neurosci. source

      Not checked in the free run.

    • cites [69] Monosov, 2020, How outcome uncertainty mediates attention, learning, and decision-making, Trends Neurosci. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Several studies found that activating serotonin neurons lengthened waiting for reward or increased active reward seeking.

    “These studies demonstrated that activating serotonin neurons increased waiting times for or active seeking of reward.”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [86] Miyazaki, 2011, Activation of dorsal raphe serotonin neurons underlies waiting for delayed rewards, J. Neurosci. source

      Not checked in the free run.

    • cites [94] Miyazaki, 2014, Optogenetic activation of dorsal raphe serotonin neurons enhances patience for future rewards, Curr. Biol. source

      Not checked in the free run.

    • cites [95] Fonseca, 2015, Activation of dorsal raphe serotonergic neurons promotes waiting but is not reinforcing, Curr. Biol. source

      Not checked in the free run.

    • cites [96] Lottem, 2018, Activation of serotonin neurons promotes active persistence in a probabilistic foraging task, Nat. Commun. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In more recent mouse work, activating serotonin neurons raised the learning rate when outcomes were separated by longer intervals, whereas learning after shorter intervals was already saturated.

    “More recent work in mice demonstrated that serotonin neuron activation increased the learning rate after longer intervals between outcomes but that learning after shorter intervals was already effectively saturated (i.e., win-stay, lose-shift).42”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [42] Iigaya, 2018, An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals, Nat. Commun. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Earlier recordings from dorsal raphe neurons, including identified serotonin neurons, showed that their activity related to the expected value of cues or contexts across multiple timescales.

    “Previous recordings from dorsal raphe neurons generally and identified serotonin neurons demonstrated a relationship between their activity and the expected value of cues or contexts on different timescales.31–33,85–87”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [31] Cohen, 2015, Serotonergic neurons signal reward and punishment on multiple timescales, eLife source

      Not checked in the free run.

    • cites [32] Li, 2016, Serotonin neurons in the dorsal raphe nucleus encode reward signals, Nat. Commun. source

      Not checked in the free run.

    • cites [33] Matias, 2017, Activity patterns of serotonin neurons underlying cognitive flexibility, eLife source

      Not checked in the free run.

    • cites [85] Nakamura, 2008, Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus, J. Neurosci. source

      Not checked in the free run.

    • cites [86] Miyazaki, 2011, Activation of dorsal raphe serotonin neurons underlies waiting for delayed rewards, J. Neurosci. source

      Not checked in the free run.

    • cites [87] Li, 2013, Association with reward negatively modulates short latency phasic conditioned responses of dorsal raphe nucleus neurons in freely moving rats, J. Neurosci. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The meta-learning RL model offers a possible mechanism for how recent experience adjusts learning, and it uncovers a quantitative relationship between serotonin neuron activity and flexible behavior.

    “The meta-learning RL model provides a potential mechanism by which recent experience modulates learning adaptively, and reveals a quantitative link between serotonin neuron activity and flexible behavior.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    For electrophysiology, the optic-fiber-tetrode bundle's placement in dorsal raphe was verified from the electrolytic lesion site.

    “In electrophysiological experiments, we confirmed targeting of the optic-fiber-tetrode bundle to the dorsal raphe by location of the electrolytic lesion.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    A neuron was classified as optogenetically identified when brief laser pulses evoked responses with short latency, low latency variability, and high probability across stimulus trains.

    “Individual neurons were determined to be optogenetically-identified if they responded to brief pulses (10 ms) of laser stimulation (473 nm wavelength) with short latency, small latency variability, and high probability of response across trains of stimulation (10 trains of 10 pulses delivered at 10 Hz).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Activity in prefrontal cortex, ventral tegmental area, and striatum tracks latent decision variables updated by experience, offering a possible route by which serotonin could influence learning.

    “The activity in these regions also correlates with latent decision variables that update with each experience,22,43,97,98,99 providing a potential substrate through which serotonin could modulate learning.”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [22] Massi, 2018, Volatility facilitates value updating in the prefrontal cortex, Neuron source

      Not checked in the free run.

    • cites [43] Bari, 2019, Stable representations of decision variables for flexible behavior, Neuron source

      Not checked in the free run.

    • cites [97] Schultz, 1997, A neural substrate of prediction and reward, Science source

      Not checked in the free run.

    • cites [98] Samejima, 2005, Representation of action-specific reward values in the striatum, Science source

      Not checked in the free run.

    • cites [99] Lau, 2008, Value representations in the primate striatum during matching behavior, Neuron source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The predictive accuracy of the behavioral models was compared using two-fold cross-validation.

    “We used two-fold cross-validation in order to compare the predictive accuracy of the behavioral models.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Explaining foraging behavior would require a change-detection element in the computation of state value so that learning can adapt.

    “To explain foraging behavior, there would need to be some change detection component to the state value computation in order to drive learning adaptively.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    To address potential statistical confounds when correlating two variables with similar autocorrelation functions, in particular serotonin neuron firing rates and expected uncertainty dynamics, each variable was simulated and compared with the real data.

    “To control for potential statistical confounds in correlating two variables with similar autocorrelation functions—in particular, firing rates of serotonin neurons and dynamics of expected uncertainty—we simulated each variable and compared it to the real data.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    Only neurons satisfying isolation criteria-L-ratio below 0.05 and fewer than 0.1% of interspike intervals under 2 ms-were included.

    “We used two metrics of isolation quality as inclusion criteria: L-ratio (< 0.05)116 and fraction of interspike interval violations (< 0.1% interspike intervals < 2 ms).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [116] Schmitzer-Torbert, 2005, Quantitative measures of cluster quality for use in extracellular recordings, Neuroscience source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Dopamine neurons in the ventral tegmental area have reward prediction error gain that varies with reward-value variance.

    “For example, the gain of RPE signals produced by dopamine neurons in the ventral tegmental area is modulated by the variance of reward value.100,101”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [100] Fiorillo, 2003, Discrete coding of reward probability and uncertainty by dopamine neurons, Science source

      Not checked in the free run.

    • cites [101] Tobler, 2005, Adaptive coding of reward value by dopamine neurons, Science source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Hierarchical Bayesian models with mouse-level hyperparameters and partial pooling were used to reduce overfitting to session noise while preserving session-to-session variability.

    “Stan was used to construct hierarchical models with mouse-level hyperparameters to govern session-level parameters. This hierarchical construction uses partial pooling to mitigate overfitting to noise in individual sessions (often seen in the point estimates for session-level parameters that result from other methods of estimation) without ignoring meaningful session-to-session variability.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Individual mice behave similarly across days.
    • The authors state this as an assumption: The learning rate needs to update faster than expected uncertainty so that change can be detected.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    In comparisons of inter-trial interval firing rates to model variables, monotonic session-wide changes in firing rates and model variables were regressed out separately before relating the residuals.

    “To control for the effect of these dynamics in comparisons of inter-trial interval firing rates to model variables, we regressed out the monotonic effects for each term separately, then regressed the firing rate residuals on the expected uncertainty residuals.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    Viral construct expression was confirmed by epifluorescence and TPH2 immunostaining.

    “We validated expression of rAAV5-EF1a-DIO-hChR2(H134R)-EYFP, pAAV-hSyn-DIO-hM4D(Gi)-mCherry, or pAAV-hSyn-DIO-mCherry with epifluorescence images of dorsal raphe (Zeiss Axio Zoom.V16) with immunostaining against tryptophan hydroxylase-2 (goat α-TPH2, Abcam 121020, at 1:400) as a marker of serotonin neurons and donkey α-goat conjugated to Cy5 (Abcam 6566) as a secondary antibody.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Such results support a role for serotonin neuron activity in monitoring expected uncertainty and promoting learning from outcomes worse than expected.

    “These findings are consistent with a role for serotonin neuron activity in tracking expected uncertainty and driving learning from worse-than-expected outcomes.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Dorsal raphe targeting was verified using electrolytic lesions followed by histology.

    “In addition to the presence of identified serotonin neurons, targeting of dorsal raphe was confirmed by performing electrolytic lesions of the tissue (20 s of 20 μA direct current across two wires of the same tetrode) and examining the tissue after perfusion.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    In each case, the animals can be viewed as learning from the absence of reward at each moment.

    “In all cases, animals can be thought of as learning from lack of rewards at each point in time.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    Unsupervised k-means clustering found four clusters; the 66 neurons in the cluster with the highest mean response probability, shortest mean latency, and smallest latency standard deviation were treated as identified.

    “We used an unsupervised k-means clustering algorithm to cluster all neurons based on these features. The elbow method and Calinkski-Harabasz criterion were used to determine that the optimal number of clusters was 4. Members of the cluster (66 neurons) with the highest mean probability of response, shortest mean latency, and smallest mean latency standard deviation were considered as identified.”
    Quote in the paperSubstantial claim
49 of 49

  • The paper relies on this without support.

    The Sert-Cre driver line and optogenetic tagging procedure specifically identify recorded dorsal raphe neurons as serotonin neurons.

    “To identify serotonin neurons, we expressed the light-gated ion channel channelrhodopsin-2 under the control of the serotonin transporter promoter in Slc6a4-Cre (also known as Sert-Cre) mice (Figures 3A and S4A).”

  • The paper relies on this without support.

    Fixing expected and unexpected uncertainty to zero in the model is a valid simulation of chemogenetic inhibition of serotonin neurons.

    “We then simulated a lesion by fixing expected and unexpected uncertainty to 0 (essentially fixing the negative RPE learning rate to its median value) and simulated behavior again (Figure 7B).”

  • The authors state this as an assumption.

    Individual mice behave similarly across days.

    “These mouse-level hyperparameters were chosen to achieve model convergence under the assumption that individual mice behave similarly across days.”

  • The authors state this as an assumption.

    The learning rate needs to update faster than expected uncertainty so that change can be detected.

    “The ordering operated under the assumption that learning rate should be integrated more quickly than expected uncertainty in order to detect change.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Neurons in MBD nuclei encode reward prediction errors.

    “Neurons in MBD nuclei are believed to encode reward prediction errors (RPEs).45”

    From Two prefrontal-subcortical circuits estimate controllability and reflect its impact on learning (2026) · cites this paper as [46]

    The Results section on serotonin neuron firing rates reports recordings from dorsal raphe neurons and links their activity to uncertainty and reward outcomes, not to reward prediction error coding in MBD nuclei. Searches for “MBD,” “midbrain dopamine,” and neuronal reward prediction error coding found no passage in our copy addressing that claim; the paper’s use of RPEs in a behavioral model does not establish it. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

  • All the neuromodulatory systems with their origins in VTA, SN, DRN, LC and VSN have at one time or another been proposed to reflect uncertainty or the potential for information gain.

    “all the neuromodulatory systems with their origins in the ventral tegmental area (VTA), substantia nigra (SN), dorsal raphe nucleus (DRN), locus coeruleus (LC) and ventral septal nucleus (VSN) have at one time or another been proposed to reflect uncertainty or the potential for information gain3,21–27”

    From Interpretable abstractions of artificial neural networks predict behavior and neural activity during human information gathering (2026) · cites this paper as [24]

  • All major neuromodulatory systems have been linked to uncertainty processing, but identifying a specific or preeminent contribution has been difficult.

    “All major neuromodulatory systems have at one time or another been linked to uncertainty processing3,21–27, but identifying a specific or preeminent contribution has been difficult.”

    From Interpretable abstractions of artificial neural networks predict behavior and neural activity during human information gathering (2026) · cites this paper as [24]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Pearce-Hall update has the opposite stated response to large errors

    Minor

    With a positive update rate, the printed equation lowers α when prediction-error magnitude exceeds α. That opposes the accompanying claim that large recent errors enhance learning rates.

  • Half of recorded foraging neurons are called a majority

    Minor

    The results report an expected-uncertainty correlation in exactly 33 of 66 foraging neurons, or 50%. The discussion calls that group a majority, which requires more than half.

  • DREADD mouse count does not match the stated exclusion

    Minor

    The methods describe six mice in each of two groups and exclude one, leaving 11 mice. Elsewhere, the final-stage task description says all DREADD-experiment mice numbered 10.

  • Mouse-group counts exceed the stated total

    Minor

    The listed groups contain 4, 5, 44, 4, and 12 mice, totaling 69 rather than the stated 67. The text does not identify an overlap between groups that would reconcile the counts.

    Reported 67 · recomputed 69

4 of 4

The citation lists haven’t been fetched yet

References 120 · Cited by 164

Authors
Grossman, C. D., Bari, B. A., & Cohen, J. Y.
Affiliations
Johns Hopkins University (US)
Corresponding author
Cohen, Jeremiah Y.
ORCID
Venue
Current Biology
Publisher
Elsevier BV
Published
2022-02
Volume
32
Issue
3
Pages
586-599.e7
ISSN
0960-9822 (ISSN-L)
DOI
10.1016/j.cub.2021.12.006
PMC
PMC8825708
OpenAlex
W4225773355
Cited by
164
References
120
Citations by year
2026: 39 · 2025: 35 · 2024: 32 · 2023: 31 · 2022: 19 · 2021: 8
FWCI
10.37
Type
journal article
Language
en
Open access
Licence
Locations
  • Current Biology (publishedVersion)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Alliance for Research on Schizophrenia and Depression
  • National Institute on Drug Abuse (R01DA042038)
  • National Institute of Neurological Disorders and Stroke (P30NS050274, R01NS104834)
Topics
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Keywords
serotonergic neurons, dorsal raphe nucleus, learning rate, uncertainty, meta-learning, decision-making, foraging behavior, classical conditioning, reversible inhibition, mice, action-outcome learning, environmental stability
MeSH terms
Animals, Learning, Neurons, Serotonin, Uncertainty, Mice, Dorsal Raphe Nucleus

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review