Separate mesocortical and mesolimbic pathways encode effort and reward learning signals
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
Significance: Learning about multiple features of a choice option is crucial for optimal decision making. How such multiattribute learning is realized remains unclear. Using functional MRI, we show that the brain exploits separate mesolimbic and mesocortical networks to simultaneously learn about reward and effort attributes. We show a double dissociation, evident in the expression of effort learning signals in dorsomedial prefrontal and reward learning in ventral striatal areas, with this dissociation being spatially mirrored in dopaminergic midbrain. At the time of choice, these segregated signals are integrated in ventral striatum. These findings highlight how the brain parses parallel learning demands.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A fast diffeomorphic image registration algorithm
Ashburner, J. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.07.007 · Cited by 8,122
Dissociable intrinsic connectivity networks for salience processing and executive control
Seeley, W. W., Menon, V., Schatzberg, A. F., Keller, J., Glover, G. H., Kenna, H., Reiss, A. L., & Greicius, M. D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5587-06.2007 · Cited by 7,426 · cited 1×
“Interestingly, when analyzing the unsigned salience PEs, we found that both effort and reward salience PEs elicit responses in regions typical for a salience network (44), and a conjunction analysis across the two salience PEs showed common activation in left anterior insula and intraparietal sulcus (SI Appendix, Fig.”
BackgroundFriston, K. J., Harrison, L., & Penny, W. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00202-7 · Cited by 4,583
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487 · cited 1×
“05 to correct for multiple comparisons (if not stated otherwise) using settings that do not show any biases in discovering false positives (90, 91).”
BackgroundDopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · cited 1×
“External support for an invigorating function comes from related work on dopamine showing that it broadcasts a motivational signal (16, 71) that in turn influences vigor (72–74).”
BackgroundA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“Note that our definition of reward PE deviates from standard notation (10, 11, 32), dictated in part by our design.”
BackgroundDopamine neuron systems in the brain: An update
Björklund, A., & Dunnett, S. B. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.006 · Cited by 1,777 · cited 3×
“By contrast, the ventrolateral region expressing effort PE activation is likely to be related to ventral tier dopamine neurons (46, 55) that form a mesocortical network targeting dmPFC and surrounding areas (46, 47).”
BackgroundDopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Self-citationTemporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Self-citationBayesian model selection for group studies
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.03.025 · Cited by 1,423
The mysterious motivational functions of mesolimbic dopamine
Salamone, J. D., & Correa, M. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.021 · Cited by 1,418 · cited 1×
“External support for an invigorating function comes from related work on dopamine showing that it broadcasts a motivational signal (16, 71) that in turn influences vigor (72–74).”
BackgroundAdaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · cited 1×
“Dopamine neurons in SN/VTA encode reward PEs (11, 48) that are broadcast to VS (12).”
BackgroundTonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133
Imaging structural co-variance between human brain regions
Alexander-Bloch, A., Giedd, J. N., & Bullmore, E. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3465 · Cited by 1,064 · cited 2×
“This means that subjects with a strong expression of a dmPFC gray matter density should also express greater gray matter density in ventrolateral SN/VTA, possibly reflecting genetic, developmental, or environmental influences (59).”
BackgroundMethodDopamine-deficient mice are severely hypoactive, adipsic, and aphagic
Zhou, Q.-Y., & Palmiter, R. D. (1995) · Cell · doi:10.1016/0092-8674(95)90145-0 · Cited by 731 · cited 2×
“Dopamine is also thought to provide a motivational signal (14–18), while dopaminergic deficits in rodents impair how effort and reward are arbitrated (1, 4, 19).”
BackgroundBOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661
Kolling, N., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2012) · Science · doi:10.1126/science.1216930 · Cited by 629
Structural covariance in the human cortex
Mechelli, A., Friston, K. J., Frackowiak, R. S., & Price, C. J. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0357-05.2005 · Cited by 606
Separate neural pathways process different decision costs
Rudebeck, P. H., Walton, M. E., Smyth, A. N., Bannerman, D. M., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1756 · Cited by 580
Effort-Based Cost–Benefit Valuation and the Human Brain
Croxson, P. L., Walton, M. E., O'Reilly, J. X., Behrens, T. E. J., & Rushworth, M. F. S. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.4515-08.2009 · Cited by 560
Prefrontal cortical regulation of brainwide circuit dynamics and reward-related behavior
Ferenczi, E. A., Zalocusky, K. A., Liston, C., Grosenick, L., Warden, M. R., Amatya, D., Katovich, K., Mehta, H., Patenaude, B., Ramakrishnan, C., et al. (2016) · Science · doi:10.1126/science.aac9698 · Cited by 535 · cited 1×
“These reward prediction errors (PEs) arise from dopaminergic neurons in substantia nigra and ventral tegmental area (SN/VTA) and are broadcast to ventral striatum (VS) to mediate reward-related adaptation and learning (12, 13).”
BackgroundA computational and neural model of momentary subjective well-being
Rutledge, R. B., Skandali, N., Dayan, P., & Dolan, R. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1407535111 · Cited by 444
Self-citationSlifstein, M., van de Giessen, E., Van Snellenberg, J., Thompson, J. L., Narendran, R., Gil, R., Hackett, E., Girgis, R., Ojeil, N., Moore, H., et al. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.2414 · Cited by 398 · cited 1×
“A long-standing hypothesis assumes an imbalance between striatal and cortical dopamine (26, 60, 61), involving excess dopamine release in striatal (62, 63) but deficient dopamine release in cortical areas (64).”
BackgroundThe role of rat medial frontal cortex in effort-based decision making
Walton, M. E., Bannerman, D. M., & Rushworth, M. F. S. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-24-10996.2002 · Cited by 364
Dopamine Synthesis Capacity Before Onset of Psychosis: A Prospective [ 18 F]-DOPA PET Imaging Study
Howes, O. D., Bose, S. K., Turkheimer, F., Valli, I., Egerton, A., Valmaggia, L. R., Murray, R. M., & McGuire, P. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11010160 · Cited by 343
A Dual Role for Prediction Error in Associative Learning
den Ouden, H. E. M., Friston, K. J., Daw, N. D., McIntosh, A. R., & Stephan, K. E. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn161 · Cited by 306
Striatal Prediction Error Modulates Cortical Coupling
den Ouden, H. E. M., Daunizeau, J., Roiser, J., Friston, K. J., & Stephan, K. E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4458-09.2010 · Cited by 304
Dopamine, Time, and Impulsivity in Humans
Pine, A., Shiner, T., Seymour, B., & Dolan, R. J. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.6028-09.2010 · Cited by 300
Self-citationCallaghan, M. F., Freund, P., Draganski, B., Anderson, E., Cappelletti, M., Chowdhury, R., Diedrichsen, J., FitzGerald, T. H. B., Smittenaar, P., Helms, G., et al. (2014) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2014.02.008 · Cited by 292
Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors
Voon, V., Pessiglione, M., Brezing, C., Gallea, C., Fernandez, H. H., Dolan, R. J., & Hallett, M. (2010) · Neuron · doi:10.1016/j.neuron.2009.12.027 · Cited by 280
Self-citationThe will to persevere Induced by electrical stimulation of the human cingulate gyrus
Parvizi, J., Rangarajan, V., Shirer, W. R., Desai, N., & Greicius, M. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.10.057 · Cited by 254
Serotonin Selectively Modulates Reward Value in Human Decision-Making
Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0053-12.2012 · Cited by 252
Self-citationTesting the Reward Prediction Error Hypothesis with an Axiomatic Model
Rutledge, R. B., Dean, M., Caplin, A., & Glimcher, P. W. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1747-10.2010 · Cited by 244
Calculating utility: preclinical evidence for cost–benefit analysis by mesolimbic dopamine
Phillips, P. E. M., Walton, M. E., & Jhou, T. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0626-6 · Cited by 237
Haber, S. N., & Behrens, T. E. J. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.031 · Cited by 230
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Self-citationFunctional imaging of the human dopaminergic midbrain
Düzel, E., Bunzeck, N., Guitart-Masip, M., Wittmann, B., Schott, B. H., & Tobler, P. N. (2009) · Trends in Neurosciences · doi:10.1016/j.tins.2009.02.005 · Cited by 216 · cited 2×
“We used the mean structural MT image where SN/ VTA can be easily distinguished from surrounding areas as a bright white stripe (45), similar to previous studies (92, 93).”
BackgroundMethodHauser, T. U., Iannaccone, R., Walitza, S., Brandeis, D., & Brem, S. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2014.09.018 · Cited by 203
Self-citationSkvortsova, V., Palminteri, S., & Pessiglione, M. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.1350-14.2014 · Cited by 177 · cited 5×
“This, together with a known involvement of dmPFC in effort-related processes (24, 25, 40), renders it likely that the same pathways are exploited in unidimensional learning contexts.”
BackgroundMethodResultInfluentialParabolic discounting of monetary rewards by physical effort
Hartmann, M. N., Hager, O. M., Tobler, P. N., & Kaiser, S. (2013) · Behavioural Processes · doi:10.1016/j.beproc.2013.09.014 · Cited by 169
Klein-Flügge, M. C., Kennerley, S. W., Saraiva, A. C., Penny, W. D., & Bestmann, S. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004116 · Cited by 163 · cited 1×
“Finally, as we are not interested in a subjective, embodied experience of ongoing force execution, we visualized the executed effort by means of a thermometer, an approach used in previous studies (39, 40).”
MethodKrebs, R. M., Heipertz, D., Schuetze, H., & Duzel, E. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.06.038 · Cited by 135 · cited 3×
“In SN/VTA, a dorsomedial encoding of reward PEs, and a ventrolateral encoding of effort PEs, extends previous studies on SN/VTA subregions (56, 57, 67, 68) by demonstrating that this segregation has functional implications that are exploited during multiattribute learning.”
BackgroundResultHauser, T. U., Moutoussis, M., Iannaccone, R., Brem, S., Walitza, S., Drechsler, R., Dayan, P., & Dolan, R. J. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005440 · Cited by 85
Self-citationScholl, J., Kolling, N., Nelissen, N., Wittmann, M. K., Harmer, C. J., & Rushworth, M. F. S. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0396-15.2015 · Cited by 85
Dopamine neurons learn to encode the long-term value of multiple future rewards
Enomoto, K., Matsumoto, N., Nakai, S., Satoh, T., Sato, T. K., Ueda, Y., Inokawa, H., Haruno, M., & Kimura, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014457108 · Cited by 83
Striatal structure and function predict individual biases in learning to avoid pain
Eldar, E., Hauser, T. U., Dayan, P., & Dolan, R. J. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1519829113 · Cited by 79 · cited 1×
“We used the segmented, normalized gray matter MT maps and applied the Jacobian of the normalization step to preserve total tissue volume (103), as reported in a previous study (84).”
MethodSelf-citation
Cited by
Motor and emotional behaviours elicited by electrical stimulation of the human cingulate cortex
Caruana, F., Gerbella, M., Avanzini, P., Gozzo, F., Pelliccia, V., Mai, R., Abdollahi, R. O., Cardinale, F., Sartori, I., Lo Russo, G., et al. (2018) · Brain · doi:10.1093/brain/awy219 · Cited by 266
Computational mechanisms of curiosity and goal-directed exploration
Schwartenbeck, P., Passecker, J., Hauser, T. U., FitzGerald, T. H., Kronbichler, M., & Friston, K. J. (2019) · eLife · doi:10.7554/elife.41703 · Cited by 254
Self-citationDevelopment, wiring and function of dopamine neuron subtypes
Garritsen, O., van Battum, E. Y., Grossouw, L. M., & Pasterkamp, R. J. (2023) · Nature reviews. Neuroscience · doi:10.1038/s41583-022-00669-3 · Cited by 128
Hird, E. J., Slanina-Davies, A., Lewis, G., Hamer, M., & Roiser, J. P. (2024) · Translational Psychiatry · doi:10.1038/s41398-024-02922-y · Cited by 79
“As well as the aMCC and AI, activity in the striatum, ventral tegmental area and posterior parietal cortex have been found to signal actual and expected effort [90 – 93].”
BackgroundZhang, T., He, K., Bai, T., Lv, H., Xie, X., Nie, J., Xie, W., Zhu, C., Wang, K., & Tian, Y. (2021) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2020.110193 · Cited by 44
Metadata
- Authors
- Hauser, T. U., Eldar, E., & Dolan, R. J.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2017-08-14
- Volume
- 114
- Issue
- 35
- Pages
- E7395-E7404
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2746265935
- Cited by
- 145
- References
- 47
- FWCI
- 6.87
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.