The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Midbrain Dopamine Neurons Signal Belief in Choice Accuracy during a Perceptual Decision
Open accessAbstract
Central to the organization of behavior is the ability to predict the values of outcomes to guide choices. The accuracy of such predictions is honed by a teaching signal that indicates how incorrect a prediction was ("reward prediction error," RPE). In several reinforcement learning contexts, such as Pavlovian conditioning and decisions guided by reward history, this RPE signal is provided by midbrain dopamine neurons. In many situations, however, the stimuli predictive of outcomes are perceptually ambiguous. Perceptual uncertainty is known to influence choices, but it has been unclear whether or how dopamine neurons factor it into their teaching signal. To cope with uncertainty, we extended a reinforcement learning model with a belief state about the perceptually ambiguous stimulus; this model generates an estimate of the probability of choice correctness, termed decision confidence. We show that dopamine responses in monkeys performing a perceptually ambiguous decision task comply with the model's predictions. Consequently, dopamine responses did not simply reflect a stimulus' average expected reward value but were predictive of the trial-to-trial fluctuations in perceptual accuracy. These confidence-dependent dopamine responses emerged prior to monkeys' choice initiation, raising the possibility that dopamine impacts impending decisions, in addition to encoding a post-decision teaching signal. Finally, by manipulating reward size, we found that dopamine neurons reflect both the upcoming reward size and the confidence in achieving it. Together, our results show that dopamine responses convey teaching signals that are also appropriate for perceptual decisions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous experimental work has shown that the phasic activity of midbrain dopamine neurons resembles the RPEs of TD learning in partially observable environments, where TD learning is performed on belief states rather than observations [6][7][8][9][10][11].”
From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [8]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133
Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027
A causal link between prediction errors, dopamine neurons and learning
Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K. A., & Janak, P. H. (2013) · Nature Neuroscience · doi:10.1038/nn.3413 · Cited by 932
Mesolimbic dopamine signals the value of work
Hamid, A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Weele, C. M. V., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2015) · Nature Neuroscience · doi:10.1038/nn.4173 · Cited by 848
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809
Rapid signalling in distinct dopaminergic axons during locomotion and reward
Howe, M. W., & Dombeck, D. A. (2016) · Nature · doi:10.1038/nature18942 · Cited by 618
Neural Basis of Reinforcement Learning and Decision Making
Lee, D., Seo, H., & Jung, M. W. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062111-150512 · Cited by 581
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Confidence and certainty: Distinct probabilistic quantities for different goals
Pouget, A., Drugowitsch, J., & Kepecs, A. (2016) · Nature Neuroscience · doi:10.1038/nn.4240 · Cited by 528
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503
Dopamine: generalization and bonuses
Kakade, S. M., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 495
The computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 492
Central Cholinergic Neurons Are Rapidly Recruited by Reinforcement Feedback
Hangya, B., Ranade, S. P., Lorenc, M., & Kepecs, A. (2015) · Cell · doi:10.1016/j.cell.2015.07.057 · Cited by 399
Serotonergic neurons signal reward and punishment on multiple timescales
Cohen, J. Y., Amoroso, M. W., & Uchida, N. (2015) · eLife · doi:10.7554/elife.06346 · Cited by 388
Lewis, D. A., Melchitzky, D. S., Sesack, S. R., Whitehead, R. E., Auh, S., & Sampson, A. (2001) · Journal of Comparative Neurology · doi:10.1002/cne.1092 · Cited by 352
Caudate Encodes Multiple Computations for Perceptual Decisions
Ding, L., & Gold, J. I. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.2894-10.2010 · Cited by 299
The basal ganglia: from motor commands to the control of vigor
Dudman, J. T., & Krakauer, J. W. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.02.005 · Cited by 287
Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280
Dopamine Neurons Can Represent Context-Dependent Prediction Error
Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y., & Hikosaka, O. (2004) · Neuron · doi:10.1016/s0896-6273(03)00869-9 · Cited by 276
Dopamine prediction error responses integrate subjective value from different reward dimensions
Lak, A., Stauffer, W. R., & Schultz, W. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1321596111 · Cited by 257
Self-citationSignatures of a Statistical Computation in the Human Sense of Confidence
Sanders, J. I., Hangya, B., & Kepecs, A. (2016) · Neuron · doi:10.1016/j.neuron.2016.03.025 · Cited by 253
Dopamine Modulates Reward-Related Vigor
Beierholm, U., Guitart‐Masip, M., Economides, M., Chowdhury, R., Düzel, E., Dolan, R. J., & Dayan, P. (2013) · Neuropsychopharmacology · doi:10.1038/npp.2013.48 · Cited by 227
Responses of pulvinar neurons reflect a subject's confidence in visual categorization
Komura, Y., Nikkuni, A., Hirashima, N., Uetake, T., & Miyamoto, A. (2013) · Nature Neuroscience · doi:10.1038/nn.3393 · Cited by 226
Orbitofrontal Cortex Is Required for Optimal Waiting Based on Decision Confidence
Lak, A., Costa, G. M., Romberg, E., Koulakov, A. A., Mainen, Z. F., & Kepecs, A. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.039 · Cited by 209
Self-citationThe Basal Ganglia’s Contributions to Perceptual Decision Making
Ding, L., & Gold, J. I. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.042 · Cited by 208
Dopamine Reward Prediction Error Responses Reflect Marginal Utility
Stauffer, W. R., Lak, A., & Schultz, W. (2014) · Current Biology · doi:10.1016/j.cub.2014.08.064 · Cited by 204
Self-citationTwo Dimensions of Value: Dopamine Neurons Represent Reward But Not Aversiveness
Fiorillo, C. D. (2013) · Science · doi:10.1126/science.1238699 · Cited by 203
Rorie, A. E., Gao, J., McClelland, J. L., & Newsome, W. T. (2010) · PLoS ONE · doi:10.1371/journal.pone.0009308 · Cited by 200
Rao, R. P. N. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00146 · Cited by 196
A Pallidus-Habenula-Dopamine Pathway Signals Inferred Stimulus Values
Bromberg-Martin, E. S., Matsumoto, M., Hong, S., & Hikosaka, O. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00158.2010 · Cited by 186
Representation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173
Summerfield, C., & Tsetsos, K. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00070 · Cited by 167
Temporally Extended Dopamine Responses to Perceptually Demanding Reward-Predictive Stimuli
Nomoto, K., Schultz, W., Watanabe, T., & Sakagami, M. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4828-09.2010 · Cited by 158
Self-citationMidbrain dopamine neurons compute inferred and cached value prediction errors in a common framework
Sadacca, B. F., Jones, J. L., & Schoenbaum, G. M. (2016) · eLife · doi:10.7554/elife.13665 · Cited by 136
Decision-Making, Errors, and Confidence in the Brain
Rolls, E. T., Fabian Grabenhorst, B., & Deco, G. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00571.2010 · Cited by 123
Distinct Representations of Cognitive and Motivational Signals in Midbrain Dopamine Neurons
Matsumoto, M., & Takada, M. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.002 · Cited by 120
Implicit knowledge of visual uncertainty guides decisions with asymmetric outcomes
Whiteley, L., & Sahani, M. (2008) · Journal of Vision · doi:10.1167/8.3.2 · Cited by 120
Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions
de Lafuente, V., & Romo, R. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1117636108 · Cited by 117
A Mathematical Framework for Statistical Decision Confidence
Hangya, B., Sanders, J. I., & Kepecs, Á. (2016) · Neural Computation · doi:10.1162/neco_a_00864 · Cited by 107
Dopamine neurons learn relative chosen value from probabilistic rewards
Lak, A., Stauffer, W. R., & Schultz, W. (2016) · eLife · doi:10.7554/elife.18044 · Cited by 88
Self-citationComplementary neural correlates of motivation in dopaminergic and noradrenergic neurons of monkeys
Bouret, S., Ravel, S., & Richmond, B. J. (2012) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2012.00040 · Cited by 73
Lloyd, K., & Dayan, P. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004622 · Cited by 51
Teichert, T., & Ferrera, V. P. (2010) · Frontiers in Neuroscience · doi:10.3389/fnins.2010.00186 · Cited by 50
Flagged first, then by citations
Cited by
Sorted by citations
Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons
Engelhard, B., Finkelstein, J., Cox, J., Fleming, W., Jang, H. J., Ornelas, S., Koay, S. A., Thiberge, S. Y., Daw, N. D., Tank, D. W., et al. (2019) · Nature · doi:10.1038/s41586-019-1261-9 · Cited by 544
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236
Metacognition and Confidence: A Review and Synthesis
Fleming, S. M. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-022423-032425 · Cited by 256
“Within a reinforcement learning framework, one perspective on such signals is that they reflect proxies for reward prediction errors driven not by external feedback, but by internal levels of choice confidence (Guggenmos et al., 2016, Lak et al., 2017).”
Striatal dopamine mediates hallucination-like perception in mice
Schmack, K., Bosc, M., Ott, T., Sturgill, J. F., & Kepecs, Á. (2021) · Science · doi:10.1126/science.abf4740 · Cited by 266
Mesolimbic dopamine release conveys causal associations
Jeong, H., Taylor, A., Floeder, J. R., Lohmann, M., Mihalaş, Ş., Wu, B., Zhou, M., Burke, D. A., & Namboodiri, V. M. K. (2022) · Science · doi:10.1126/science.abq6740 · Cited by 232
Metadata
- Authors
- Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A.
- Affiliations
- Cold Spring Harbor Laboratory (US)
- Tamagawa University (JP)
- Gatsby Computational Neuroscience Unit (GB)
- Venue
- Current Biology
- Publisher
- Elsevier BV
- Published
- 2017-03
- Volume
- 27
- Issue
- 6
- Pages
- 821-832
- ISSN
- 0960-9822 (ISSN-L)
- PMC
- PMC5819757
- OpenAlex
- W2592093412
- Cited by
- 170
- References
- 52
- Citations by year
- 2026: 14 · 2025: 16 · 2024: 22 · 2023: 19 · 2022: 9 · 2021: 16 · 2020: 24 · 2019: 22 · 2018: 20 · 2017: 8
- FWCI
- 9.90
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Current Biology (publishedVersion)
- PubMed (publishedVersion)
- City Research Online (City University London) (acceptedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Cold Spring Harbor Laboratory Institutional Repository (Cold Spring Harbor Laboratory) (acceptedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Funders
- Sir Henry Wellcome Trust Postdoctoral Fellowship (106101/Z/14/Z)
- Swartz Foundation
- National Institute of Mental Health (NIH) (R01MH097061)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- midbrain dopaminergic neurons, reward prediction error, perceptual decision making, perceptual uncertainty, decision confidence, belief states, reinforcement learning models, perceptual ambiguity, reward magnitude, teaching signal, choice accuracy
- MeSH terms
- Animals, Choice Behavior, Decision Making, Dopamine, Macaca, Male, Mesencephalon, Perception, Reinforcement, Psychology, Reward, Models, Animal, Dopaminergic Neurons
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
