Two Anatomically and Computationally Distinct Learning Signals Predict Changes to Stimulus-Outcome Associations in Hippocampus
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
Complex cognitive processes require sophisticated local processing but also interactions between distant brain regions. It is therefore critical to be able to study distant interactions between local computations and the neural representations they act on. Here we report two anatomically and computationally distinct learning signals in lateral orbitofrontal cortex (lOFC) and the dopaminergic ventral midbrain (VM) that predict trial-by-trial changes to a basic internal model in hippocampus. To measure local computations during learning and their interaction with neural representations, we coupled computational fMRI with trial-by-trial fMRI suppression. We find that suppression in a medial temporal lobe network changes trial-by-trial in proportion to stimulus-outcome associations. During interleaved choice trials, we identify learning signals that relate to outcome type in lOFC and to reward value in VM. These intervening choice feedback signals predicted the subsequent change to hippocampal suppression, suggesting a convergence of signals that update the flexible representation of stimulus-outcome associations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Repetition and the brain: neural models of stimulus-specific effects
Grill-Spector, K., Henson, R., & Martin, A. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.11.006 · Cited by 2,256 · cited 1×
“Different mechanistic accounts have been advanced to explain RS, including fatigue, sparse coding, and predictive coding (Grill-Spector et al., 2006; Summerfield et al., 2008; Wiggs and Martin, 1998).”
BackgroundA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“Motivated by an extensive literature implicating the dopaminerich VM in updating beliefs (Klein-Flügge et al., 2011; Montague et al., 1996), we interrogated the BOLD response in VM (ROIs) (defined independently using coordinates from Klein-Flügge et al.”
MethodLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Self-citationThe hippocampus and memory: insights from spatial processing
Bird, C. M., & Burgess, N. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2335 · Cited by 1,313
Bayesian surprise attracts human attention
Itti, L., & Baldi, P. (2009) · Vision Research · doi:10.1016/j.visres.2008.09.007 · Cited by 1,224
Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys
Carmichael, S. T., & Price, J. L. (1995) · Journal of Comparative Neurology · doi:10.1002/cne.903630408 · Cited by 1,197 · cited 2×
“…and provides a major neocortical input into hippocampus (Bird and Burgess, 2008) and amygdala, which is reciprocally connected to OFC in macaques (Carmichael and Price, 1995; Stefanacci and Amaral, 2002) and whose functional interactions with OFC have been the topic of active investigation…”
BackgroundGläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Frontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111
Self-citationDistinct roles for direct and indirect pathway striatal neurons in reinforcement
Kravitz, A. V., Tye, L. D., & Kreitzer, A. C. (2012) · Nature Neuroscience · doi:10.1038/nn.3100 · Cited by 972
Expectation (and attention) in visual cognition
Summerfield, C., & Egner, T. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.06.003 · Cited by 950 · cited 2×
“…the relationship we observed between suppression and association strength include predictive coding of outcomes elicited by stimulus presentation (Summerfield and Egner, 2009) and/or plasticity between the underlying neuronal populations that encode a particular stimulus and a particular…”
BackgroundOptimal decision making and the anterior cingulate cortex
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1724 · Cited by 948 · cited 2×
“…sulcal subdivision in the context of instrumental reward value learning (Behrens et al., 2007) and perceptual choice (O’Reilly et al., 2013), complementing evidence that lesions to this region in macaques produce impairments in the appropriate integration of past reward (Kennerley et al., 2006).”
BackgroundSelf-citationOrbitofrontal Cortex as a Cognitive Map of Task Space
Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.005 · Cited by 885 · cited 6×
“…2011; Noonan et al., 2012; Rudebeck and Murray, 2014; Rudebeck et al., 2013b; Rushworth et al., 2011; Stalnaker et al., 2014; Takahashi et al., 2011; Walton et al., 2010; Wilson et al., 2014), supporting the hypothesis that lOFC may be important for updating beliefs about likely reward outcomes.”
BackgroundInfluentialPeters, J., & Büchel, C. (2010) · Neuron · doi:10.1016/j.neuron.2010.03.026 · Cited by 871 · cited 1×
“While most previous research has focused on hippocampal interactions withmedial prefrontal cortex, notably when associative information is used to evaluate or imagine choices online (Barron et al., 2013; Kumaran et al., 2009; Peters and Büchel, 2010; Wikenheiser and Redish, 2015), lOFC-hippocampal interactions have be…”
BackgroundHare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792
Wittmann, B. C., Schott, B. H., Guderian, S., Frey, J. U., Heinze, H.-J., & Düzel, E. (2005) · Neuron · doi:10.1016/j.neuron.2005.01.010 · Cited by 698 · cited 2×
“…the other hand, have direct projections to hippocampus proper (Gasbarri et al., 1994), and learningrelated coupling between these structures has previously been discovered in the context of facilitating generalization and long-termmemory formation (Shohamy andWagner, 2008; Wittmann et al., 2005).”
BackgroundNeural repetition suppression reflects fulfilled perceptual expectations
Summerfield, C., Trittschuh, E. H., Monti, J. M., Mesulam, M.-M., & Egner, T. (2008) · Nature Neuroscience · doi:10.1038/nn.2163 · Cited by 692 · cited 1×
“Different mechanistic accounts have been advanced to explain RS, including fatigue, sparse coding, and predictive coding (Grill-Spector et al., 2006; Summerfield et al., 2008; Wiggs and Martin, 1998).”
BackgroundBoorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665 · cited 1×
“…neighboring subdivision within ventromedial PFC, that have been shown to encode abstract values and comparisons between goal values during choices (Boorman et al., 2009, 2013b; Chib et al., 2009; Howard et al., 2015; Lim et al., 2011). lOFC, extending into VLPFC, was one region in a network that…”
BackgroundSelf-citationBOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661
Chib, V. S., Rangel, A., Shimojo, S., & O'Doherty, J. P. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2575-09.2009 · Cited by 643 · cited 2×
“Importantly, the rostrolateral localization of the OFC update signal in our study can be contrasted with signals in more medial subdivisions of OFC, and neighboring subdivision within ventromedial PFC, that have been shown to encode abstract values and comparisons between goal values during choices (Boorman et al., 20…”
BackgroundResultGremel, C. M., & Costa, R. M. (2013) · Nature Communications · doi:10.1038/ncomms3264 · Cited by 618 · cited 4×
“Previous studies have pointed to lOFC involvement in learning and/or using choice-outcome associations to guide behavior (Buckley et al., 2009; Gremel and Costa, 2013; Jones et al., 2012; Noonan et al., 2011; Rudebeck and Murray, 2014; Rudebeck et al., 2013b; Rushworth et al., 2011; Takahashi et al., 2011; Walton et a…”
BackgroundInfluentialWhat the orbitofrontal cortex does not do
Stalnaker, T. A., Cooch, N. K., & Schoenbaum, G. (2015) · Nature Neuroscience · doi:10.1038/nn.3982 · Cited by 531 · cited 2×
“…but was not sensitive to the online stimulus-outcome association strength during CSS blocks, suggests that lOFC may utilize an internal model to update beliefs about, or facilitate decisions between, rewarding goals (Stalnaker et al., 2015; Wilson et al., 2014) but may not store the model locally.”
BackgroundRobust group analysis using outlier inference
Woolrich, M. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.02.042 · Cited by 504 · cited 1×
“To detect and de-weight outliers, we performed robust group analysis using outlier inference, applying FEAT’s outlier de-weighting option (Woolrich, 2008).”
MethodHampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498
The Computation of Social Behavior
Behrens, T. E. J., Hunt, L. T., & Rushworth, M. F. S. (2009) · Science · doi:10.1126/science.1169694 · Cited by 407 · cited 1×
“…computations, such as prediction errors, volatility or uncertainty estimates, and volatility prediction errors, across several domains of cognition (Behrens et al., 2007, 2009; Boorman et al., 2013a; Cooper et al., 2010; D’Ardenne et al., 2008; Daw et al., 2011; den Ouden et al., 2009; Hampton et…”
BackgroundSelf-citationRudebeck, P. H., & Murray, E. A. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.049 · Cited by 405 · cited 4×
“Previous studies have pointed to lOFC involvement in learning and/or using choice-outcome associations to guide behavior (Buckley et al., 2009; Gremel and Costa, 2013; Jones et al., 2012; Noonan et al., 2011; Rudebeck and Murray, 2014; Rudebeck et al., 2013b; Rushworth et al., 2011; Takahashi et al., 2011; Walton et a…”
BackgroundInfluentialWalton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397
Self-citationPrefrontal mechanisms of behavioral flexibility, emotion regulation and value updating
Rudebeck, P. H., Saunders, R. C., Prescott, A. T., Chau, L. S., & Murray, E. A. (2013) · Nature Neuroscience · doi:10.1038/nn.3440 · Cited by 383
Some observations on cortical inputs to the macaque monkey amygdala: An anterograde tracing study
Stefanacci, L., & Amaral, D. G. (2002) · Journal of Comparative Neurology · doi:10.1002/cne.10339 · Cited by 345
Lim, S.-L., O'Doherty, J. P., & Rangel, A. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.1246-11.2011 · Cited by 329 · cited 2×
“Importantly, the rostrolateral localization of the OFC update signal in our study can be contrasted with signals in more medial subdivisions of OFC, and neighboring subdivision within ventromedial PFC, that have been shown to encode abstract values and comparisons between goal values during choices (Boorman et al., 20…”
BackgroundResultStatistical learning of visual transitions in monkey inferotemporal cortex
Meyer, T., & Olson, C. R. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1112895108 · Cited by 325 · cited 4×
“…of increased suppression for associated, compared to non-associated stimulus-reward or stimulusstimulus pairs, albeit in the absence of any online learning (Klein-Flügge et al., 2013; Meyer and Olson, 2011), we predicted a reduction in the BOLD response for HC items when compared to LC items.”
BackgroundResultInfluentialDissociable components of rule-guided behavior depend on distinct medial and prefrontal regions
Buckley, M. J., Mansouri, F. A., Hoda, H., Mahboubi, M., Browning, P. G. F., Kwok, S. C., Phillips, A., & Tanaka, K. (2009) · Science · doi:10.1126/science.1172377 · Cited by 319 · cited 2×
“…and recording evidence across species suggests a key role for lOFC in learning and/or using stimulus-outcome identity associations to guide choice (Buckley et al., 2009; Gremel and Costa, 2013; Jones et al., 2012; McDannald et al., 2011; Noonan et al., 2012; Rudebeck and Murray, 2014; Rudebeck et…”
BackgroundMapping value based planning and extensively trained choice in the human brain
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3068 · Cited by 310
A Dual Role for Prediction Error in Associative Learning
den Ouden, H. E. M., Friston, K. J., Daw, N. D., McIntosh, A. R., & Stephan, K. E. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn161 · Cited by 306 · cited 1×
“…several domains of cognition (Behrens et al., 2007, 2009; Boorman et al., 2013a; Cooper et al., 2010; D’Ardenne et al., 2008; Daw et al., 2011; den Ouden et al., 2009; Hampton et al., 2006; Hare et al., 2008; Iglesias et al., 2013; Klein-Flügge et al., 2011; Payzan-LeNestour and Bossaerts,…”
BackgroundExpectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280 · cited 5×
“…learning and/or using choice-outcome associations to guide behavior (Buckley et al., 2009; Gremel and Costa, 2013; Jones et al., 2012; Noonan et al., 2011; Rudebeck and Murray, 2014; Rudebeck et al., 2013b; Rushworth et al., 2011; Takahashi et al., 2011; Walton et al., 2010; Wilson et al., 2014).”
BackgroundInfluentialJocham, G., Klein, T. A., & Ullsperger, M. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3904-10.2011 · Cited by 278 · cited 1×
“However, despite the computational insight bestowed from identifying one type of learning signal or another, outside of striatal dopamine (Collins and Frank, 2014; Jocham et al., 2011; Kravitz et al., 2012), little is known about how these different signals are used in the brain.”
BackgroundRisk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267 · cited 2×
“…several domains of cognition (Behrens et al., 2007, 2009; Boorman et al., 2013a; Cooper et al., 2010; D’Ardenne et al., 2008; Daw et al., 2011; den Ouden et al., 2009; Hampton et al., 2006; Hare et al., 2008; Iglesias et al., 2013; Klein-Flügge et al., 2011; Payzan-LeNestour and Bossaerts, 2011).”
BackgroundTracking the emergence of conceptual knowledge during human decision making
Kumaran, D., Summerfield, J. J., Hassabis, D., & Maguire, E. A. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.030 · Cited by 260 · cited 2×
“…interactions withmedial prefrontal cortex, notably when associative information is used to evaluate or imagine choices online (Barron et al., 2013; Kumaran et al., 2009; Peters and Büchel, 2010; Wikenheiser and Redish, 2015), lOFC-hippocampal interactions have been relatively unexplored.”
Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement
Noonan, M. P., Kolling, N., Walton, M. E., & Rushworth, M. F. S. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08023.x · Cited by 169 · cited 2×
“…stimulus-outcome identity associations to guide choice (Buckley et al., 2009; Gremel and Costa, 2013; Jones et al., 2012; McDannald et al., 2011; Noonan et al., 2012; Rudebeck and Murray, 2014; Rudebeck et al., 2013b; Rushworth et al., 2011; Stalnaker et al., 2014; Takahashi et al., 2011;…”
BackgroundDissociable reward and timing signals in human midbrain and ventral striatum
Klein-Flügge, M. C., Hunt, L. T., Bach, D. R., Dolan, R. J., & Behrens, T. E. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.024 · Cited by 75 · cited 4×
“Motivated by an extensive literature implicating the dopaminerich VM in updating beliefs (Klein-Flügge et al., 2011; Montague et al., 1996), we interrogated the BOLD response in VM (ROIs) (defined independently using coordinates from Klein-Flügge et al.”
BackgroundMethodInfluentialSelf-citation
Cited by
What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior
Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.002 · Cited by 1,055
Self-citationLearning task-state representations
Niv, Y. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0470-8 · Cited by 356
A map of abstract relational knowledge in the human hippocampal-entorhinal cortex
Garvert, M. M., Dolan, R. J., & Behrens, T. E. (2017) · eLife · doi:10.7554/elife.17086 · Cited by 379
Self-citationPredictive representations can link model-based reinforcement learning to model-free mechanisms
Russek, E. M., Momennejad, I., Botvinick, M. M., Gershman, S. J., & Daw, N. D. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005768 · Cited by 326
Map Making: Constructing, Combining, and Inferring on Abstract Cognitive Maps
Park, S. A., Miller, D. S., Nili, H., Ranganath, C., & Boorman, E. D. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.030 · Cited by 270
Self-citation
Metadata
- Authors
- Boorman, E. D., Rajendran, V. G., O’Reilly, J. X., & Behrens, T. E.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2016-03
- Volume
- 89
- Issue
- 6
- Pages
- 1343-1354
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W2288411222
- Cited by
- 117
- References
- 39
- FWCI
- 9.99
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.