The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Dopamine transients are sufficient and necessary for acquisition of model-based associations
Open accessAbstract
Associative learning is driven by prediction errors. Dopamine transients correlate with these errors, which current interpretations limit to endowing cues with a scalar quantity reflecting the value of future rewards. We tested whether dopamine might act more broadly to support learning of an associative model of the environment. Using sensory preconditioning, we show that prediction errors underlying stimulus–stimulus learning can be blocked behaviorally and reinstated by optogenetically activating dopamine neurons. We further show that suppressing the firing of these neurons across the transition prevents normal stimulus–stimulus learning. These results establish that the acquisition of model-based information about transitions between nonrewarding events is also driven by prediction errors and that, contrary to existing canon, dopamine transients are both sufficient and necessary to support this type of learning. Our findings open new possibilities for how these biological signals might support associative learning in the mammalian brain in these and other contexts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Cognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationModel-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Toward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318
Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning
Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., de Lecea, L., & Deisseroth, K. (2009) · Science · doi:10.1126/science.1168878 · Cited by 1,266
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Dopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,117
Dopamine reward prediction-error signalling: A two-component response
Schultz, W. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.26 · Cited by 1,036
Motivational control of goal-directed action
Dickinson, A., & Balleine, B. (1994) · Animal Learning & Behavior · doi:10.3758/bf03199951 · Cited by 995
Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system
Lammel, S., Hetzel, A., Häckel, O., Jones, I., Liss, B., & Roeper, J. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.022 · Cited by 976
"Attention-like" processes in classical conditioning
Kamin, L. J. (1967) · NASA Technical Reports Server (NASA) · Cited by 938
A causal link between prediction errors, dopamine neurons and learning
Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K., & Janak, P. H. (2013) · Nature Neuroscience · doi:10.1038/nn.3413 · Cited by 869
Dopamine neurons and their role in reward mechanisms
Schultz, W. (1997) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(97)80007-4 · Cited by 766
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710
Witten, I. B., Steinberg, E. E., Lee, S. Y., Davidson, T. J., Zalocusky, K. A., Brodsky, M., Yizhar, O., Cho, S. L., Gong, S., Ramakrishnan, C., et al. (2011) · Neuron · doi:10.1016/j.neuron.2011.10.028 · Cited by 680
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661
Conditioned stimulus as a determinant of the form of the Pavlovian conditioned response.
Holland, P. C. (1977) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037//0097-7403.3.1.77 · Cited by 561
Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions
Wimmer, G. E., & Shohamy, D. (2012) · Science · doi:10.1126/science.1223252 · Cited by 515
Dopamine: generalization and bonuses
Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 480
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429
Associations in second-order conditioning and sensory preconditioning.
Rizley, R. C., & Rescorla, R. A. (1972) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0033333 · Cited by 420
Brogden, W. J. (1939) · Journal of Experimental Psychology · doi:10.1037/h0058944 · Cited by 409
Arithmetic and local circuitry underlying dopamine prediction errors
Eshel, N., Bukwich, M., Rao, V., Hemmelder, V., Tian, J., & Uchida, N. (2015) · Nature · doi:10.1038/nature14855 · Cited by 392
Relations between Pavlovian-instrumental transfer and reinforcer devaluation
Holland, P. C. (2004) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/0097-7403.30.2.104 · Cited by 388
Adamantidis, A. R., Tsai, H.-C., Boutrel, B., Zhang, F., Stuber, G. D., Budygin, E. A., Tourino, C., Bonci, A., Deisseroth, K., & de Lecea, L. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.2246-11.2011 · Cited by 357
Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target
Parker, N. F., Cameron, C. M., Taliaferro, J. P., Lee, J., Choi, J. Y., Davidson, T. J., Daw, N. D., & Witten, I. B. (2016) · Nature Neuroscience · doi:10.1038/nn.4287 · Cited by 355
Holland, P. C., & Rescorla, R. A. (1975) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037//0097-7403.1.4.355 · Cited by 350
Orbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values
Jones, J. L., Esber, G. R., McDannald, M. A., Gruber, A. J., Hernandez, A., Mirenzi, A., & Schoenbaum, G. (2012) · Science · doi:10.1126/science.1227489 · Cited by 335
Self-citationDopamine neurons share common response function for reward prediction error
Eshel, N., Tian, J., Bukwich, M., & Uchida, N. (2016) · Nature Neuroscience · doi:10.1038/nn.4239 · Cited by 316
Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm
Tobler, P. N., Dickinson, A., & Schultz, W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-32-10402.2003 · Cited by 300
Takahashi, Y. K., Roesch, M. R., Stalnaker, T. A., Haney, R. Z., Calu, D. J., Taylor, A. R., Burke, K. A., & Schoenbaum, G. (2009) · Neuron · doi:10.1016/j.neuron.2009.03.005 · Cited by 284
Self-citationBurst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat
Horvitz, J. C., Stewart, T., & Jacobs, B. L. (1997) · Brain Research · doi:10.1016/s0006-8993(97)00265-5 · Cited by 283
Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion
Ilango, A., Kesner, A. J., Keller, K. L., Stuber, G. D., Bonci, A., & Ikemoto, S. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.1703-13.2014 · Cited by 255
Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1417219112 · Cited by 233
McDannald, M. A., Lucantonio, F., Burke, K. A., Niv, Y., & Schoenbaum, G. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.5499-10.2011 · Cited by 214
Self-citationBrief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors
Chang, C. Y., Esber, G. R., Marrero-Garcia, Y., Yau, H.-J., Bonci, A., & Schoenbaum, G. (2016) · Nature Neuroscience · doi:10.1038/nn.4191 · Cited by 206
Self-citationA Pallidus-Habenula-Dopamine Pathway Signals Inferred Stimulus Values
Bromberg-Martin, E. S., Matsumoto, M., Hong, S., & Hikosaka, O. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00158.2010 · Cited by 186
The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards
Burke, K. A., Franz, T. M., Miller, D. N., & Schoenbaum, G. (2008) · Nature · doi:10.1038/nature06993 · Cited by 180
Self-citationEsber, G. R., & Haselgrove, M. (2011) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2011.0836 · Cited by 178
Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework
Sadacca, B. F., Jones, J. L., & Schoenbaum, G. M. (2016) · eLife · doi:10.7554/elife.13665 · Cited by 146
Self-citationOverriding Phasic Dopamine Signals Redirects Action Selection during Risk/Reward Decision Making
Stopper, C. M., Tse, M. T. L., Montes, D. R., Wiedman, C. R., & Floresco, S. B. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.033 · Cited by 136
Sensory preconditioning of human subjects.
Brogden, W. J. (1947) · Journal of Experimental Psychology · doi:10.1037/h0058465 · Cited by 125
Aitken, T. J., Greenfield, V. Y., & Wassum, K. M. (2015) · Journal of Neurochemistry · doi:10.1111/jnc.13494 · Cited by 123
Chemogenetic Silencing of Neurons in Retrosplenial Cortex Disrupts Sensory Preconditioning
Robinson, S., Todd, T. P., Pasternak, A. R., Luikart, B. W., Skelton, P. D., Urban, D. J., & Bucci, D. J. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.1349-14.2014 · Cited by 122
Effects of amygdala central nucleus lesions on blocking and unblocking.
Holland, P. C., & Gallagher, M. (1993) · Behavioral Neuroscience · doi:10.1037//0735-7044.107.2.235 · Cited by 117
Preserved Sensitivity to Outcome Value after Lesions of the Basolateral Amygdala
Blundell-Birtill, P., Hall, G., & Killcross, S. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-20-07702.2003 · Cited by 93
Phasic dopamine release in the medial prefrontal cortex enhances stimulus discrimination
Popescu, A. T., Zhou, M., & Poo, M. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1606098113 · Cited by 93
Physiological state gates acquisition and expression of mesolimbic reward prediction signals
Cone, J. J., Fortin, S. M., McHenry, J. A., Stuber, G. D., McCutcheon, J. E., & Roitman, M. F. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1519643113 · Cited by 77
Looking for cognition in the structure within the noise
Johnson, A. C., Fenton, A. A., Kentros, C. G., & Redish, A. D. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.11.005 · Cited by 68
The Prelimbic Cortex Contributes to the Down-Regulation of Attention Toward Redundant Cues
Sharpe, M. J., & Killcross, S. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhs393 · Cited by 54
Self-citationVariations in Unconditioned Stimulus Processing in Unblocking.
Holland, P. C., & Kenmuir, C. L. (2005) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.31.2.155 · Cited by 37
An Associative Analysis of Instrumental Learning
Colwill, R. M. (1993) · Current Directions in Psychological Science · doi:10.1111/1467-8721.ep10772598 · Cited by 33
Multiplexing signals in reinforcement learning with internal models and dopamine
Nakahara, H. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.01.001 · Cited by 33
Cited by
Reinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667
Striatal circuits for reward learning and decision-making
Cox, J., & Witten, I. B. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0189-2 · Cited by 568
Hallucinations and strong priors
Corlett, P. R., Horga, G., Fletcher, P. C., Alderson-Day, B., Schmack, K., & Powers, A. R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.001 · Cited by 534
Prioritized memory access explains planning and hippocampal replay
Mattar, M. G., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0232-z · Cited by 393
Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties
Saunders, B. T., Richard, J. M., Margolis, E. B., & Janak, P. H. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0191-4 · Cited by 374
Metadata
- Authors
- Sharpe, M. J., Chang, C. Y., Liu, M. A., Batchelor, H. M., Mueller, L. E., Jones, J. L., Niv, Y., & Schoenbaum, G.
- Venue
- Nature Neuroscience
- Published
- 2017-04-03
- Volume
- 20
- Issue
- 5
- Pages
- 735-742
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2604220916
- Cited by
- 300
- References
- 57
- FWCI
- 13.50
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.