The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment
Open accessAbstract
Summary Significant evidence supports the view that dopamine shapes learning by encoding reward prediction errors. However, it is unknown whether striatal targets receive tailored dopamine dynamics based on regional functional specialization. Here, we report wave-like spatiotemporal activity-patterns in dopamine axons and release across the dorsal striatum. These waves switch between activational motifs and organize dopamine transients into localized clusters within functionally related striatal subregions. Notably, wave trajectories were tailored to task demands, propagating from dorsomedial to dorsolateral striatum when rewards are contingent on animal behavior, and in the opponent direction when rewards are independent of behavioral responses. We propose a computational architecture in which striatal dopamine waves are sculpted by inference about agency, and provide a mechanism to direct credit assignment to specialized striatal subregions. Supporting model predictions, dorsomedial dopamine activity during reward-pursuit signaled the extent of instrumental control, and interacted with reward waves to predict future behavioral adjustments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Ultrasensitive fluorescent proteins for imaging neuronal activity
Chen, T.-W., Wardill, T. J., Sun, Y., Pulver, S. R., Renninger, S. L., Baohan, A., Schreiter, E. R., Kerr, R. A., Orger, M. B., Jayaraman, V., et al. (2013) · Nature · doi:10.1038/nature12354 · Cited by 6,474
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
Functional anatomy of the basal ganglia. I. The cortico-basal ganglia-thalamo-cortical loop
Parent, A., & Hazrati, L.-N. (1995) · Brain Research Reviews · doi:10.1016/0165-0173(94)00007-c · Cited by 2,161
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“According to the reward prediction error (RPE) hypothesis, transients in DA signaling reflect deviations from reward expectation that drive reinforcement learning (RL) (Montague et al., 1996; Schultz et al., 1997).”
Habits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · cited 6×
“An important clue is the functional architecture of hierarchical corticostriatal loops (Graybiel, 2008; Haber, 2003), wherein multiple striatal ‘‘actors’’ (or subregions) gate the selection of cortical actions at various functional levels of abstraction (Balleine et al., 2015; Frank, 2011).”
InfluentialThe primate basal ganglia: Parallel and integrative networks
Haber, S. N. (2003) · Journal of Chemical Neuroanatomy · doi:10.1016/j.jchemneu.2003.10.003 · Cited by 1,479 · cited 4×
“If regional heterogeneity is an adaptive feature of striatal DA dynamics, what are the organizational rules for large-scale DA transmission, and how do they facilitate computational/circuit operations in the service of behavioral flexibility? An important clue is the functional architecture of hierarchical corticostri…”
InfluentialThe Role of the Dorsal Striatum in Reward and Decision-Making: Figure 1.
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1554-07.2007 · Cited by 1,390
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955
Berke, J. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0152-y · Cited by 904
Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits
Lerner, T. N., Shilyansky, C., Davidson, T. J., Evans, K. E., Beier, K. T., Zalocusky, K. A., Crow, A. K., Malenka, R. C., Luo, L., Tomer, R., et al. (2015) · Cell · doi:10.1016/j.cell.2015.07.014 · Cited by 794 · cited 1×
“One hypothesis motivated by the excitationrelease coupling principle in neurobiology would suggest that DA waves may be inherited from the sequential firing of topographically projectingmidbrain DA cells (Lerner et al., 2015).”
Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops
Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004) · Nature Neuroscience · doi:10.1038/nn1279 · Cited by 779
A critical time window for dopamine actions on the structural plasticity of dendritic spines
Yagishita, S., Hayashi-Takagi, A., Ellis-Davies, G. C. R., Urakubo, H., Ishii, S., & Kasai, H. (2014) · Science · doi:10.1126/science.1255514 · Cited by 614
Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574
Prolonged dopamine signalling in striatum signals proximity and value of distant rewards
Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M., & Graybiel, A. M. (2013) · Nature · doi:10.1038/nature12475 · Cited by 565
Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons
Engelhard, B., Finkelstein, J., Cox, J., Fleming, W., Jang, H. J., Ornelas, S., Koay, S. A., Thiberge, S. Y., Daw, N. D., Tank, D. W., et al. (2019) · Nature · doi:10.1038/s41586-019-1261-9 · Cited by 544
A comprehensive excitatory input map of the striatum reveals novel functional organization
Hunnicutt, B. J., Jongbloets, B. C., Birdsong, W. T., Gertz, K. J., Zhong, H., & Mao, T. (2016) · eLife · doi:10.7554/elife.19103 · Cited by 536
DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429
Cognitive control over learning: Creating, clustering, and generalizing task-set structure.
Collins, A. G. E., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0030852 · Cited by 426
Self-citationRedish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346 · cited 1×
“Notably, this interpretation hints at a different role for sRPEs (facilitating inference about responsible actors) compared with the large RPEs following reward itself (facilitating reinforcement learning): a dual operation that can also be gated (Franklin and Frank, 2015; Gershman et al., 2015; Redish et al., 2007; S…”
Dopamine enhances model-based over model-free choice behavior
Wunderlich, K., Smittenaar, P., & Dolan, R. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.042 · Cited by 294 · cited 1×
“Inactivation or manipulation of DA in DMS degrades goal-directed planning and action due to an inability to learn whether rewards are under instrumental control (Balleine and O’Doherty, 2010; Wunderlich et al., 2012).”
BackgroundPrensa, L., & Parent, A. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-18-07247.2001 · Cited by 238 · cited 1×
“First, DA axon projections to the forebrain are extensively divergent (Matsuda et al., 2009; Prensa and Parent, 2001), providing an architecture for broadcast-like communication.”
BackgroundRepresentation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173 · cited 1×
“Thesimulationcontained a bank of parallel agents representing striatal subregions (Frank and Badre, 2012; Figure S5), and tone/state transitions formulated as sequential semi-Markov states (Daw et al., 2006).”
Continuous and Discrete Neuron Types of the Adult Murine Striatum
Stanley, G., Gokce, O., Malenka, R. C., Südhof, T. C., & Quake, S. R. (2020) · Neuron · doi:10.1016/j.neuron.2019.11.004 · Cited by 158 · cited 1×
“The model captures regional reward credit assignment in functionally specialized cortico-basal ganglia (BG) loops, inspired by previous anatomical and functional reports (Aoki et al., 2019; Barbera et al., 2016; Haber, 2003; Hintiryan et al., 2016; Hunnicutt et al., 2016; Klaus et al., 2017; Lee et al., 2020; Mandelba…”
Discovering latent causes in reinforcement learning
Gershman, S. J., Norman, K. A., & Niv, Y. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.07.007 · Cited by 151 · cited 2×
“Notably, this interpretation hints at a different role for sRPEs (facilitating inference about responsible actors) compared with the large RPEs following reward itself (facilitating reinforcement learning): a dual operation that can also be gated (Franklin and Frank, 2015; Gershman et al., 2015; Redish et al., 2007; S…”
Stimulus representation and the timing of reward-prediction errors in models of the dopamine system
Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2008) · Neural Computation · doi:10.1162/neco.2008.11-07-654 · Cited by 148
Gershman, S. J., Pesaran, B., & Daw, N. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2469-09.2009 · Cited by 118 · cited 1×
“While such regional, actor-specific striatal RPEs are reported in human fMRI studies (Badre and Frank, 2012; Gershman et al., 2009), we currently lack an empirical demonstration of whether DA signals are tailored to subregions according to their”
Cited by
Geometric constraints on human brain function
Pang, J. C., Aquino, K. M., Oldehinkel, M., Robinson, P. A., Fulcher, B. D., Breakspear, M., & Fornito, A. (2023) · Nature · doi:10.1038/s41586-023-06098-1 · Cited by 440
A Unified Framework for Dopamine Signals across Timescales
Kim, H. R., Malik, A. N., Mikhael, J. G., Bech, P., Tsutsui-Kimura, I., Sun, F., Zhang, Y., Li, Y., Watabe-Uchida, M., Gershman, S. J., et al. (2020) · Cell · doi:10.1016/j.cell.2020.11.013 · Cited by 302
Spatial and temporal scales of dopamine transmission
Liu, C., Goel, P., & Kaeser, P. S. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-021-00455-7 · Cited by 301
Stress-sensitive inference of task controllability
Ligneul, R., Mainen, Z. F., Ly, V., & Cools, R. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01306-w · Cited by 60
Watanabe, H., Dijkstra, J. M., & Nagatsu, T. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25042009 · Cited by 48
Metadata
- Authors
- Hamid, A. A., Frank, M. J., & Moore, C. I.
- Venue
- Cell
- Publisher
- Elsevier BV
- Published
- 2021-05
- Volume
- 184
- Issue
- 10
- Pages
- 2733-2749.e16
- ISSN
- 0092-8674 (ISSN-L)
- OpenAlex
- W3156105972
- Cited by
- 226
- References
- 30
- FWCI
- 14.71
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.