The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
The Medial Prefrontal Cortex Shapes Dopamine Reward Prediction Errors under State Uncertainty
Open accessAbstract
Animals make predictions based on currently available information. In natural settings, sensory cues may not reveal complete information, requiring the animal to infer the "hidden state" of the environment. The brain structures important in hidden state inference remain unknown. A previous study showed that midbrain dopamine neurons exhibit distinct response patterns depending on whether reward is delivered in 100% (task 1) or 90% of trials (task 2) in a classical conditioning task. Here we found that inactivation of the medial prefrontal cortex (mPFC) affected dopaminergic signaling in task 2, in which the hidden state must be inferred ("will reward come or not?"), but not in task 1, where the state was known with certainty. Computational modeling suggests that the effects of inactivation are best explained by a circuit in which the mPFC conveys inference over hidden states to the dopamine system. VIDEO ABSTRACT.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The brain may have dedicated machinery, perhaps in prefrontal cortex [12,13], for computing belief states, which could then be provided to downstream areas, such as the basal ganglia, to perform standard RL algorithms such as TD learning [14].”
From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [13]
In the citing paper
“As a working example, we will consider the probabilistic associative learning paradigm where dopamine RPEs were shown to be consistent with a belief representation [7,13].”
From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [13]
In the citing paper
“In both tasks, mice were trained to associate an odor cue with probabilistic delivery of a liquid reward 1.2-2.8slater.”
From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [13]
In the citing paper
“Previous work has shown that, in animals, prefrontal cortex activity is a necessary component of animals' state representations [13].”
From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [13]
In the citing paper
“This work found that inactivating prefrontal cortex in the Starkweather task led to animals' RPEs in Task 2 resembling the RPEs of Task 1.”
From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [13]
In the citing paper
“For instance, previous work has suggested that prefrontal cortex may perform state estimation in tasks with hidden states [6,7,12,13].”
From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
Sesack, S. R., Deutch, A. Y., Roth, R. H., & Bunney, B. S. (1989) · Journal of Comparative Neurology · doi:10.1002/cne.902900205 · Cited by 1,568
Differential projections of the infralimbic and prelimbic cortex in the rat
Vertes, R. P. (2004) · Synapse · doi:10.1002/syn.10279 · Cited by 1,550
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318
Self-citationMidbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Whole-Brain Mapping of Direct Inputs to Midbrain Dopamine Neurons
Watabe‐Uchida, M., Zhu, L., Ogawa, S. K., Vamanrao, A., & Uchida, N. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.017 · Cited by 1,277
Self-citationPrefrontal cortex in the rat: Projections to subcortical autonomic, motor, and limbic centers
Gabbott, P. L. A., Warner, T. A., Jays, P. R. L., Salway, P., & Busby, S. J. (2005) · Journal of Comparative Neurology · doi:10.1002/cne.20738 · Cited by 1,227
The hippocampus as a predictive map
Stachenfeld, K. L., Botvinick, M. M., & Gershman, S. J. (2017) · Nature Neuroscience · doi:10.1038/nn.4650 · Cited by 1,000
Self-citationOrbitofrontal Cortex as a Cognitive Map of Task Space
Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.005 · Cited by 885
Carr, D. B., & Sesack, S. R. (2000) · Journal of Neuroscience · doi:10.1523/jneurosci.20-10-03864.2000 · Cited by 880
Time cells in the hippocampus: a new dimension for mapping memories
Eichenbaum, H. (2014) · Nature Reviews Neuroscience · doi:10.1038/nrn3827 · Cited by 843
Speed and accuracy of olfactory discrimination in the rat
Uchida, N., & Mainen, Z. F. (2003) · Nature Neuroscience · doi:10.1038/nn1142 · Cited by 687
Self-citationStatistically optimal perception and learning: from behavior to neural representations
Fiser, J., Berkes, P., Orbán, G., & Lengyel, M. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.003 · Cited by 672
Atasoy, D., Aponte, Y., Su, H. H., & Sternson, S. M. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1954-08.2008 · Cited by 635
Probabilistic brains: knowns and unknowns
Pouget, A., Beck, J. M., Ma, W. J., & Latham, P. E. (2013) · Nature Neuroscience · doi:10.1038/nn.3495 · Cited by 615 · cited 2×
“Inference based on ambiguous information is a fundamental computation the brain must perform in natural environments (Fiser et al., 2010; Pouget et al., 2013).”
A representation of the hazard rate of elapsed time in macaque area LIP
Janssen, P., & Shadlen, M. N. (2005) · Nature Neuroscience · doi:10.1038/nn1386 · Cited by 601
Maintenance of persistent activity in a frontal thalamocortical loop
Guo, Z. V., Inagaki, H. K., Daie, K., Druckmann, S., Gerfen, C. R., & Svoboda, K. (2017) · Nature · doi:10.1038/nature22324 · Cited by 529
Distinct behavioural and network correlates of two interneuron types in prefrontal cortex
Kvitsiani, D., Ranade, S., Hangya, B., Taniguchi, H., Huang, J. Z., & Kepecs, A. (2013) · Nature · doi:10.1038/nature12176 · Cited by 516
Bäckman, C. M., Malik, N., Zhang, Y., Shan, L., Grinberg, A., Hoffer, B. J., Westphal, H., & Tomac, A. C. (2006) · genesis · doi:10.1002/dvg.20228 · Cited by 463
Bayesian theories of conditioning in a changing world
Courville, A. C., Daw, N. D., & Touretzky, D. S. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.004 · Cited by 457
Influence of Reward Delays on Responses of Dopamine Neurons
Kobayashi, S., & Schultz, W. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1600-08.2008 · Cited by 440
Arithmetic and local circuitry underlying dopamine prediction errors
Eshel, N., Bukwich, M., Rao, V., Hemmelder, V., Tian, J., & Uchida, N. (2015) · Nature · doi:10.1038/nature14855 · Cited by 401
Self-citationA New DREADD Facilitates the Multiplexed Chemogenetic Interrogation of Behavior
Vardy, E., Robinson, J. E., Li, C., Olsen, R. H. J., DiBerto, J. F., Giguère, P. M., Sassano, F. M., Huang, X., Zhu, H., Urban, D. J., et al. (2015) · Neuron · doi:10.1016/j.neuron.2015.03.065 · Cited by 398
A Scalable Population Code for Time in the Striatum
Mello, G. B. M. E., Soares, S., & Paton, J. J. (2015) · Current Biology · doi:10.1016/j.cub.2015.02.036 · Cited by 398
Precise olfactory responses tile the sniff cycle
Shusterman, R., Smear, M. C., Koulakov, A. A., & Rinberg, D. A. (2011) · Nature Neuroscience · doi:10.1038/nn.2877 · Cited by 391
Context, learning, and extinction
Gershman, S. J., Blei, D. M., & Niv, Y. (2010) · Psychological Review · doi:10.1037/a0017808 · Cited by 386 · cited 1×
“That is, the current state is ‘‘hidden’’ and needs to be inferred from partial information (Courville et al., 2006; Gershman et al., 2010, 2015).”
Self-citationLima, S. Q., Hromádka, T., Znamenskiy, P., & Zador, A. M. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006099 · Cited by 356
Mitchell, A. S. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.03.001 · Cited by 323
The temporal precision of reward prediction in dopamine neurons
Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008) · Nature Neuroscience · doi:10.1038/nn.2159 · Cited by 310
Vertes, R. P. (2001) · The Journal of Comparative Neurology · doi:10.1002/cne.10083 · Cited by 280
Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280
Temporal maps and informativeness in associative learning
Balsam, P. D., & Gallistel, C. R. (2009) · Trends in Neurosciences · doi:10.1016/j.tins.2008.10.004 · Cited by 264
An event map of memory space in the hippocampus
Deuker, L., Bellmund, J. L. S., Schröder, T. N., & Doeller, C. F. (2016) · eLife · doi:10.7554/elife.16534 · Cited by 243
Medial Orbitofrontal Cortex Mediates Outcome Retrieval in Partially Observable Task Situations
Bradfield, L. A., Dezfouli, A., van Holstein, M., Chieng, B. C. H., & Balleine, B. W. (2015) · Neuron · doi:10.1016/j.neuron.2015.10.044 · Cited by 211
Dopamine reward prediction errors reflect hidden-state inference across time
Starkweather, C. K., Babayan, B. M., Uchida, N., & Gershman, S. J. (2017) · Nature Neuroscience · doi:10.1038/nn.4520 · Cited by 206
Self-citationLearning of sequential movements by neural network model with dopamine-like reinforcement signal
Suri, R. E., & Schultz, W. (1998) · Experimental Brain Research · doi:10.1007/s002210050467 · Cited by 198
Rao, R. P. N. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00146 · Cited by 196
Going and stopping: dichotomies in behavioral control by the prefrontal cortex
Gourley, S. L., & Taylor, J. R. (2016) · Nature Neuroscience · doi:10.1038/nn.4275 · Cited by 195
Habenula Lesions Reveal that Multiple Mechanisms Underlie Dopamine Prediction Errors
Tian, J., & Uchida, N. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.028 · Cited by 173
Self-citationRepresentation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173
Neural Correlates of Interval Timing in Rodent Prefrontal Cortex
Kim, J., Ghim, J.-W., Lee, J. H., & Jung, M. W. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.1443-13.2013 · Cited by 170
Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision
Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A. (2017) · Current Biology · doi:10.1016/j.cub.2017.02.026 · Cited by 170
Time and space in the hippocampus
Howard, M. W., & Eichenbaum, H. (2014) · Brain Research · doi:10.1016/j.brainres.2014.10.069 · Cited by 165
Temporally Extended Dopamine Responses to Perceptually Demanding Reward-Predictive Stimuli
Nomoto, K., Schultz, W., Watanabe, T., & Sakagami, M. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4828-09.2010 · Cited by 158
Representation of interval timing by temporally scalable firing patterns in rat prefrontal cortex
Xu, M., Zhang, S., Dan, Y., & Poo, M. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1321314111 · Cited by 157
Discovering latent causes in reinforcement learning
Gershman, S. J., Norman, K. A., & Niv, Y. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.07.007 · Cited by 151
Self-citationStimulus representation and the timing of reward-prediction errors in models of the dopamine system
Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2008) · Neural Computation · doi:10.1162/neco.2008.11-07-654 · Cited by 148
Takahashi, Y. K., Langdon, A., Niv, Y., & Schoenbaum, G. M. (2016) · Neuron · doi:10.1016/j.neuron.2016.05.015 · Cited by 120
Effects of Prefrontal Cortical Inactivation on Neural Activity in the Ventral Tegmental Area
Jo, Y. S., Lee, J. J.-H., & Mizumori, S. J. Y. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.0118-13.2013 · Cited by 76
Inactivation of medial prefrontal cortex impairs time interval discrimination in rats
Kim, J. E. (2009) · Frontiers in Behavioral Neuroscience · doi:10.3389/neuro.08.038.2009 · Cited by 71
Dopamine neurons encode errors in predicting movement trigger occurrence
Pasquereau, B., & Turner, R. S. (2014) · Journal of Neurophysiology · doi:10.1152/jn.00401.2014 · Cited by 54
Prefrontal Regulation of Neuronal Activity in the Ventral Tegmental Area
Jo, Y. S., & Mizumori, S. J. Y. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhv215 · Cited by 51
Mice infer probabilistic models for timing
Li, Y., & Dudman, J. T. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1310666110 · Cited by 40
Scalar timing in memory: A temporal map in the hippocampus
Oprisan, S. A., Aft, T., Buhusi, M. C., & Buhusi, C. V. (2017) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2017.11.012 · Cited by 22
Tsunoda, Y., & Kakei, S. (2008) · Neuroscience Letters · doi:10.1016/j.neulet.2007.12.063 · Cited by 21
Flagged first, then by citations
Cited by
Sorted by citations
A Neural Circuit Mechanism for Encoding Aversive Stimuli in the Mesolimbic Dopamine System
de Jong, J. W., Afjei, S. A., Dorocic, I. P., Peck, J. R., Liu, C. J., Kim, C. K., Tian, L., Deisseroth, K. A., & Lammel, S. (2018) · Neuron · doi:10.1016/j.neuron.2018.11.005 · Cited by 525
The Relationship Between Uncertainty and Affect
Anderson, E. C., Carleton, R. N., Diefenbach, M., & Han, P. K. J. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02504 · Cited by 332
The prefrontal cortex, pathological anxiety, and anxiety disorders
Kenwood, M. M., Kalin, N. H., & Barbas, H. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01109-z · Cited by 379
Reinforcement learning in artificial and biological systems
Neftci, E. O., & Averbeck, B. B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0025-4 · Cited by 268
Dopamine release in the nucleus accumbens core signals perceived saliency
Kutlu, M. G., Zachry, J. E., Melugin, P. R., Cajigas, S. A., Chevee, M. F., Kelly, S. J., Kutlu, B., Tian, L., Siciliano, C. A., & Calipari, E. S. (2021) · Current Biology · doi:10.1016/j.cub.2021.08.052 · Cited by 238
Metadata
- Authors
- Starkweather, C. K., Gershman, S. J., & Uchida, N.
- Affiliations
- Harvard University (US)
- Corresponding author
- Gershman, Samuel J.
- Uchida, Naoshige
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2018-05
- Volume
- 98
- Issue
- 3
- Pages
- 616-629.e6
- ISSN
- 0896-6273 (ISSN-L)
- PMC
- PMC5934341
- OpenAlex
- W2797393690
- Cited by
- 136
- References
- 57
- Citations by year
- 2026: 13 · 2025: 23 · 2024: 14 · 2023: 19 · 2022: 12 · 2021: 17 · 2020: 19 · 2019: 13 · 2018: 6
- FWCI
- 6.26
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Neuron (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- NIH (R01MH095953, R01MH101207)
- CRCNS (R01-1207833, F30MH112242-01A1, T32MH020017, T32GM007753)
- Harvard Brain Science Initiative
- Simons Collaboration on Global Brain
- Harvard Mind Brain and Behavior
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- medial prefrontal cortex, dopamine, reward prediction error, state uncertainty, hidden state inference, midbrain dopaminergic neurons, neural inactivation, classical conditioning, computational modeling
- MeSH terms
- Action Potentials, Animals, Dopamine, Forecasting, Male, Mice, Inbred C57BL, Mice, Transgenic, Nerve Net, Odorants, Reward, Prefrontal Cortex, Uncertainty, Mice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
