An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals
Open accessPublished version of a preprint
Abstract
Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed two distinct choice strategies. Choices after short inter-trial-intervals (ITIs) depended only on the last trial outcome and followed a win-stay-lose-switch pattern. In contrast, choices after long ITIs reflected outcome history over multiple trials, as described by reinforcement learning models. We found that optogenetic stimulation during a trial significantly boosted the rate of learning that occurred due to the outcome of that trial, but these effects were only exhibited on choices after long ITIs. This suggests that serotonin neurons modulate reinforcement learning rates, and that this influence is masked by alternate, unaffected, decision mechanisms. These results provide insight into the role of serotonin in treating psychiatric disorders, particularly its modulation of neural plasticity and learning.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior research demonstrates that serotonin neurons modulate flexible behavior in changing environments.
“Prior research demonstrates that serotonin neurons modulate flexible behavior in changing environments.33,37–42”
From Serotonin neurons modulate learning rate through uncertainty (2022) · cites this paper as [42]
Brief excitations of serotonin neurons in a probabilistic choice task enhanced learning rates after long intervals between outcomes.
“Conversely, brief excitations of serotonin neurons in a probabilistic choice task enhanced learning rates after long intervals between outcomes.42”
From Serotonin neurons modulate learning rate through uncertainty (2022) · cites this paper as [42]
The authors' results and framework, including how learning from worse-than-expected outcomes is affected, agree with earlier observations and manipulations of serotonin neurons.
“Our findings and conceptual framework, including the effect on learning from worse-than-expected outcomes, are consistent with previous observations and manipulations of serotonin neuron activity.”
From Serotonin neurons modulate learning rate through uncertainty (2022) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Self-citationWatkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,406 · see citing works
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · see citing works
The empirical status of cognitive-behavioral therapy: A review of meta-analyses
Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006) · Clinical Psychology Review · doi:10.1016/j.cpr.2005.07.003 · Cited by 3,215 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
Self-citationHerrnstein, R. J. (1961) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1961.4-267 · Cited by 2,281 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Self-citationLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · see citing works
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · see citing works
Self-citationMatching Behavior and the Representation of Value in the Parietal Cortex
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004) · Science · doi:10.1126/science.1094765 · Cited by 1,030 · see citing works
5-HT and mechanisms of defence
Deakin, J. F. W., & Graeff, F. G. (1991) · Journal of Psychopharmacology · doi:10.1177/026988119100500414 · Cited by 1,011 · see citing works
Reconciling the role of central serotonin neurons in human and animal behavior
Soubrié, P. (1986) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00022871 · Cited by 993 · see citing works
Gong, S., Doughty, M., Harbaugh, C. R., Cummins, A., Hatten, M. E., Heintz, N., & Gerfen, C. R. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2707-07.2007 · Cited by 977 · see citing works
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · see citing works
Self-citationMetalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704 · see citing works
Prefrontal cortex and decision making in a mixed-strategy game
Barraclough, D. J., Conroy, M. L., & Lee, D. (2004) · Nature Neuroscience · doi:10.1038/nn1209 · Cited by 678 · see citing works
Neural computations underlying arbitration between model-based and model-free learning
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642 · see citing works
Hawkins, R. D., Abrams, T. W., Carew, T. J., & Kandel, E. R. (1983) · Science · doi:10.1126/science.6294833 · Cited by 617 · see citing works
Nassar, M. R., Wilson, R. C., Heasly, B. S., & Gold, J. I. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.0822-10.2010 · Cited by 564 · see citing works
Learning and selective attention
Dayan, P., Kakade, S., & Montague, P. R. (2000) · Nature Neuroscience · doi:10.1038/81504 · Cited by 495 · see citing works
Self-citationDYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485 · see citing works
Dorsal raphe neurons signal reward through 5-HT and glutamate
Liu, Z., Zhou, J., Li, Y., Hu, F., Lu, Y., Ma, M., Feng, Q., Zhang, J.-E., Wang, D., Zeng, J., et al. (2014) · Neuron · doi:10.1016/j.neuron.2014.02.010 · Cited by 469 · see citing works
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456 · see citing works
Self-citationCollins, A. G. E., & Frank, M. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07980.x · Cited by 419 · see citing works
Go and no-go learning in reward and punishment: Interactions between affect and effect
Guitart‐Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 418 · see citing works
Self-citationDisentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding
Huys, Q. J. M., Cools, R., Gölzer, M., Friedel, E., Heinz, A., Dolan, R. J., & Dayan, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002028 · Cited by 415 · see citing works
Self-citationThe dynamics of memory as a consequence of optimal adaptation to a changing body
Kording, K. P., Tenenbaum, J. B., & Shadmehr, R. (2007) · Nature Neuroscience · doi:10.1038/nn1901 · Cited by 393 · see citing works
Serotonergic neurons signal reward and punishment on multiple timescales
Cohen, J. Y., Amoroso, M. W., & Uchida, N. (2015) · eLife · doi:10.7554/elife.06346 · Cited by 388 · see citing works
Prefrontal Serotonin Depletion Affects Reversal Learning But Not Attentional Set Shifting
Clarke, H. F., Walker, S. C., Crofts, H. S., Dalley, J. W., Robbins, T. W., & Roberts, A. C. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.3690-04.2005 · Cited by 358 · see citing works
Learning in spiking neural networks by reinforcement of stochastic synaptic transmission
Seung, H. S. (2003) · Neuron · doi:10.1016/s0896-6273(03)00761-x · Cited by 340 · see citing works
Jacobs, B. L., & Fornal, C. A. (1997) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(97)80141-9 · Cited by 311 · see citing works
Hart, A. S., Rutledge, R. B., Glimcher, P. W., & Phillips, P. E. M. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.2489-13.2014 · Cited by 294 · see citing works
Optogenetic Activation of Dorsal Raphe Serotonin Neurons Enhances Patience for Future Rewards
Miyazaki, K. W., Miyazaki, K., Tanaka, K. F., Yamanaka, A., Takahashi, A., Tabuchi, S., & Doya, K. (2014) · Current Biology · doi:10.1016/j.cub.2014.07.041 · Cited by 278 · see citing works
Crockett, M. J., Clark, L., & Robbins, T. W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2513-09.2009 · Cited by 271 · see citing works
Serotonin, inhibition, and negative mood
Dayan, P., & Huys, Q. J. M. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040004 · Cited by 263 · see citing works
Self-citationSerotonin Selectively Modulates Reward Value in Human Decision-Making
Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0053-12.2012 · Cited by 252 · see citing works
Self-citationGallistel, C. R., Mark, T. A., King, A. P., & Latham, P. E. (2001) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.27.4.354 · Cited by 225 · see citing works
Activation of Dorsal Raphe Serotonergic Neurons Promotes Waiting but Is Not Reinforcing
Fonseca, M. S., Murakami, M., & Mainen, Z. F. (2015) · Current Biology · doi:10.1016/j.cub.2014.12.002 · Cited by 210 · see citing works
Self-citationLINEAR-NONLINEAR-POISSON MODELS OF PRIMATE CHOICE DYNAMICS
Corrado, G. S., Sugrue, L. P., Seung, H. S., & Newsome, W. T. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.23-05 · Cited by 206 · see citing works
Fusi, S., Asaad, W. F., Miller, E. K., & Wang, X.-J. (2007) · Neuron · doi:10.1016/j.neuron.2007.03.017 · Cited by 199 · see citing works
Deurwaerdère, P. D., & Giovanni, G. D. (2016) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2016.03.004 · Cited by 164 · see citing works
Reinstated episodic context guides sampling-based decisions for reward
Bornstein, A. M., & Norman, K. A. (2017) · Nature Neuroscience · doi:10.1038/nn.4573 · Cited by 161 · see citing works
Dopamine neurons encoding long-term memory of object value for habitual behavior
Kim, H. F., Ghazizadeh, A., & Hikosaka, O. (2015) · Cell · doi:10.1016/j.cell.2015.10.063 · Cited by 157 · see citing works
Reinforcement learning and decision making in monkeys during a competitive game
Lee, D., Conroy, M. L., McGreevy, B. P., & Barraclough, D. J. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.07.007 · Cited by 156 · see citing works
Bayesian Inference Underlies the Contraction Bias in Delayed Comparison Tasks
Ashourian, P., & Loewenstein, Y. (2011) · PLoS ONE · doi:10.1371/journal.pone.0019551 · Cited by 149 · see citing works
Vetencourt, J. F. M., Tiraboschi, E., Spolidoro, M., Castrén, E., & Maffei, L. (2011) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2010.07488.x · Cited by 138 · see citing works
Correia, P. A., Lottem, E., Banerjee, D., Machado, A. S., Carey, M. R., & Mainen, Z. F. (2017) · eLife · doi:10.7554/elife.20975 · Cited by 138 · see citing works
Self-citationWorthy, D. A., Hawthorne, M. J., & Otto, A. R. (2012) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0324-9 · Cited by 136 · see citing works
Herrnstein, R. J. (1997) · Cited by 135 · see citing works
Adding prediction risk to the theory of reward learning
PREUSCHOFF, K., & BOSSAERTS, P. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.005 · Cited by 133 · see citing works
A Biophysically Based Neural Model of Matching Law Behavior: Melioration by Stochastic Synapses
Soltani, A., & Wang, X. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.5159-05.2006 · Cited by 127 · see citing works
Sub-Second Dopamine Detection in Human Striatum
Kishida, K. T., Sandberg, S. G., Lohrenz, T. M., Comair, Y. G., Sáez, I., Phillips, P. E. M., & Montague, P. R. (2011) · PLoS ONE · doi:10.1371/journal.pone.0023291 · Cited by 123 · see citing works
The modulation of savouring by prediction error and its effects on choice
Iigaya, K., Story, G. W., Kurth-Nelson, Z. L., Dolan, R. J., & Dayan, P. (2016) · eLife · doi:10.7554/elife.13747 · Cited by 110 · see citing works
Self-citationLoewenstein, Y., & Seung, H. S. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0505220103 · Cited by 109 · see citing works
Phasic responses in dorsal raphe serotonin neurons to noxious stimuli
Schweimer, J. V., & Ungless, M. A. (2010) · Neuroscience · doi:10.1016/j.neuroscience.2010.09.058 · Cited by 102 · see citing works
Walker, S. C., Robbins, T. W., & Roberts, A. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn136 · Cited by 96 · see citing works
Fletcher, P., Korth, K. M., & Chambers, J. W. (1999) · Psychopharmacology · doi:10.1007/s002130051170 · Cited by 95 · see citing works
Serotonin: modulator of a drive to withdraw.
Tops, M., Russo, S., Boksem, M. A. S., & Tucker, D. M. (2009) · Brain and Cognition · doi:10.1016/j.bandc.2009.03.009 · Cited by 86 · see citing works
Do dorsal raphe 5-HT neurons encode “beneficialness”?
Luo, M., Li, Y., & Zhong, W. (2016) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2016.08.008 · Cited by 76 · see citing works
The Actor-Critic Learning Is Behind the Matching Law: Matching Versus Optimal Behaviors
Sakai, Y., & Fukai, T. (2007) · Neural Computation · doi:10.1162/neco.2008.20.1.227 · Cited by 55 · see citing works
Iigaya, K. (2016) · eLife · doi:10.7554/elife.18073 · Cited by 43 · see citing works
Self-citationIigaya, K., Jolivald, A., Jitkrittum, W., Gilchrist, I. D., Dayan, P., Paul, E. S., & Mendl, M. (2016) · PLoS ONE · doi:10.1371/journal.pone.0165840 · Cited by 39 · see citing works
Self-citationDorsal Raphe Serotonergic Neurons Control Intertemporal Choice under Trade-off
Xu, S., Das, G., Hueske, E., & Tonegawa, S. (2017) · Current Biology · doi:10.1016/j.cub.2017.09.008 · Cited by 37 · see citing works
Dynamics of Feature Categorization
Marti, D. N., & Rinzel, J. (2012) · Neural Computation · doi:10.1162/neco_a_00383 · Cited by 24 · see citing works
Operant Matching as a Nash Equilibrium of an Intertemporal Game
Loewenstein, Y., Prelec, D., & Seung, H. S. (2009) · Neural Computation · doi:10.1162/neco.2009.09-08-854 · Cited by 24 · see citing works
Intertrial interval duration and impulsive choice
Smethells, J. R., & Reilly, M. P. (2014) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.131 · Cited by 23 · see citing works
Uncertainty, phase and oscillatory hippocampal recall
Lengyel, M., & Dayan, P. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0109 · Cited by 16 · see citing works
Self-citationDynamical Regimes in Neural Network Models of Matching Behavior
Iigaya, K., & Fusi, S. (2013) · Neural Computation · doi:10.1162/neco_a_00522 · Cited by 15 · see citing works
Self-citationIigaya, K., Ahmadian, Y., Sugrue, L. P., Corrado, G. S., Loewenstein, Y., Newsome, W. T., & Fusi, S. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/141309 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Mice alternate between discrete strategies during perceptual decision-making
Ashwood, Z. C., Roy, N. A., Stone, I. R., Laboratory, T. I. B., Urai, A. E., Churchland, A. K., Pouget, A., & Pillow, J. W. (2022) · Nature Neuroscience · doi:10.1038/s41593-021-01007-z · Cited by 229 · see citing works
Serotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164 · see citing works
Complementary roles of serotonergic and cholinergic systems in decisions about when to act
Khalighinejad, N., Manohar, S., Husain, M., & Rushworth, M. F. S. (2022) · Current Biology · doi:10.1016/j.cub.2022.01.042 · Cited by 29 · see citing works
From reinforcement learning to agency: Frameworks for understanding basal cognition
Seifert, G., Sealander, A., Marzen, S., & Levin, M. (2024) · BioSystems · doi:10.1016/j.biosystems.2023.105107 · Cited by 16 · see citing works
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6 · see citing works
- Authors
- Iigaya, K., Fonseca, M. S., Murakami, M., Mainen, Z. F., & Dayan, P.
- Affiliations
- California Institute of Technology (US)
- Centro Champalimaud (PT)
- Gatsby Computational Neuroscience Unit (GB)
- Corresponding author
- Iigaya, Kiyohito
- Venue
- Nature Communications
- Published
- 2018-06-26
- Volume
- 9
- Issue
- 1
- Pages
- 2477-2477
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC6018802
- OpenAlex
- W2809461452
- Cited by
- 120
- References
- 72
- Citations by year
- 2026: 15 · 2025: 10 · 2024: 22 · 2023: 16 · 2022: 14 · 2021: 12 · 2020: 20 · 2019: 9 · 2018: 2
- FWCI
- 4.90
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Nature Communications (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Max Planck Digital Library (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Funders
- Gatsby Charitable Foundation
- Max-Planck-Gesellschaft
- Fundação Champalimaud
- Japan Society for the Promotion of Science
- Fundação para a Ciência e a Tecnologia (PD/BD/52446/2013)
- H2020 European Research Council (671251)
- FP7 Ideas: European Research Council (250334)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- serotonin, dorsal raphe nucleus, optogenetic stimulation, reinforcement learning, learning rate, intertrial interval, decision strategies, neuroplasticity
- MeSH terms
- Animals, Choice Behavior, Decision Making, Learning, Mice, Transgenic, Models, Neurological, Models, Psychological, Neuronal Plasticity, Reinforcement, Psychology, Reward, Serotonin, Time Factors, Serotonin Plasma Membrane Transport Proteins, Mice, Serotonergic Neurons, Optogenetics, Dorsal Raphe Nucleus
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
