More Than the Sum of Its Parts: A Role for the Hippocampus in Configural Reinforcement Learning
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Summary People often perceive configurations rather than the elements they are made of, a bias that may emerge because configurations often predict behaviorally relevant outcomes. Yet, little is known about how the brain learns to associate configurations with outcomes and how this learning differs from learning about individual elements. We combined behavior, reinforcement learning models and functional imaging in humans to understand how people learn to associate configurations of cues with outcomes. We found that configural learning varied based on the relative predictive strength of elements vs. configurations and was related to both the strength of BOLD activity in the hippocampus and patterns of hippocampal BOLD responses. Configural learning was further related to functional connectivity between the hippocampus and the nucleus accumbens. Moreover, configural learning was associated with flexible knowledge about cue-outcome associations and with differential eye movements during choice behavior. Together, these findings demonstrate that configural learning is associated with a distinct computational, cognitive and neural profile that is well-suited to support flexible and adaptive behavior.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A component based noise correction method (CompCor) for BOLD and perfusion based fMRI
Behzadi, Y., Restom, K., Liau, J., & Liu, T. T. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.04.042 · Cited by 4,502
Forest before trees: The precedence of global features in visual perception
Navon, D. (1977) · Cognitive Psychology · doi:10.1016/0010-0285(77)90012-3 · Cited by 4,005 · cited 2×
“This strategy is consistent with the observation that people and animals tend to perceive configurations rather than cues (Kimchi, 1994; Navon, 1977) and that combinations of cues are treated differently from the sum of their elements (Pearce, 1987, 1994; Rudy and Sutherland, 1995).”
Methods to detect, characterize, and remove motion artifact in resting state fMRI
Power, J. D., Mitra, A., Laumann, T. O., Snyder, A. Z., Schlaggar, B. L., & Petersen, S. E. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.08.048 · Cited by 3,748
Large-scale automated synthesis of human functional neuroimaging data.
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011) · Nature Methods · doi:10.1038/nmeth.1635 · Cited by 3,686
Representational similarity analysis-connecting the branches of systems neuroscience
Kriegeskorte, N. (2008) · Frontiers in Systems Neuroscience · doi:10.3389/neuro.06.004.2008 · Cited by 3,355
Simple memory: a theory for archicortex
Marr, D. (1971) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1971.0078 · Cited by 2,771 · cited 1×
“…the need to separate overlapping conditions for reinforcement learning parallels a similar requirement for interference reduction in hippocampal-dependent memory, namely, pattern separation between memories with overlapping elements (Marr, 1971; Norman and O’Reilly, 2003; Treves and Rolls, 1992).”
BackgroundThe Medial Temporal Lobe and Recognition Memory
Eichenbaum, H., Yonelinas, A. P., & Ranganath, C. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094328 · Cited by 2,580
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 3×
“The nucleus accumbens is known to play an important role in updating value in reinforcement learning (O’Doherty et al., 2004), and it also has strong anatomical connections with the anterior/ventral hippocampus (Groenewegen et al., 1987; Kelley and Domesick, 1982).”
BackgroundMerging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906
A neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · cited 2×
“We leveraged reinforcement learning models that have been widely used to characterize learning about a single cue (e.g., Dayan and Daw, 2008; Glimcher, 2011; O’Doherty et al., 2003; Schultz, 1998; Sutton and Barto, 1998), and we extended them to understand howpeople learn to predict outcomes based on configurations of…”
BackgroundMethodNorman, K. A., & O'Reilly, R. C. (2003) · Psychological Review · doi:10.1037/0033-295x.110.4.611 · Cited by 1,312
Information processing in the primate visual system: an integrated systems perspective
Van Essen, D. C., Anderson, C. H., & Felleman, D. J. (1992) · Science · doi:10.1126/science.1734518 · Cited by 1,266 · cited 1×
“The role of the hippocampus in associative binding is consistent with its position at the apex of the visual processing hierarchy (Amaral and Lavenex, 2006; Van Essen et al., 1992).”
BackgroundItem, context and relational episodic encoding in humans
Davachi, L. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.10.012 · Cited by 1,138
From Conditioning to Conscious Recollection
Eichenbaum, H., & Cohen, N. J. (2004) · doi:10.1093/acprof:oso/9780195178043.001.0001 · Cited by 1,026
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955 · cited 2×
“We leveraged reinforcement learning models that have been widely used to characterize learning about a single cue (e.g., Dayan and Daw, 2008; Glimcher, 2011; O’Doherty et al., 2003; Schultz, 1998; Sutton and Barto, 1998), and we extended them to understand howpeople learn to predict outcomes based on configurations of…”
MethodA simple method for distinguishing within- versus between-subject effects using mixed models
van de Pol, M., & Wright, J. (2009) · Animal Behaviour · doi:10.1016/j.anbehav.2008.11.006 · Cited by 951
Peters, J., & Büchel, C. (2010) · Neuron · doi:10.1016/j.neuron.2010.03.026 · Cited by 871
Large-scale evaluation of ANTs and FreeSurfer cortical thickness measurements
Tustison, N. J., Cook, P. A., Klein, A., Song, G., Das, S. R., Duda, J. T., Kandel, B. M., van Strien, N., Stone, J. R., Gee, J. C., et al. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.05.044 · Cited by 670 · cited 1×
“…5.0.1 FMRIB; Smith et al., 2004 https://fsl.fmrib.ox.ac.uk/fsl/fslwiki AFNI 2011.12.21.0938 Cox, 1996; https://afni.nimh.nih.gov ANTs 2.1 stnava; Tustison et al., 2014 http://stnava.github.io/ANTs/ Neurosynth Yarkoni et al., 2011 http://neurosynth.org/ R version 3.3.0 R Development Core Team,…”
InfluentialStatistical learning of new visual feature combinations by infants
Fiser, J., & Aslin, R. N. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.232472899 · Cited by 637
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Self-citationNeural representations of events arise from temporal community structure
Schapiro, A. C., Rogers, T. T., Cordova, N. I., Turk-Browne, N. B., & Botvinick, M. M. (2013) · Nature Neuroscience · doi:10.1038/nn.3331 · Cited by 556 · cited 1×
“In turn, learning statistical regularities in the co-occurrence of features can guide how features are integrated into perceptual units (Fiser and Aslin, 2002; Saffran et al., 1996) and neural representations (Schapiro et al., 2012, 2013).”
BackgroundMemory deficits for implicit contextual information in amnesic subjects with hippocampal damage
Chun, M. M., & Phelps, E. A. (1999) · Nature Neuroscience · doi:10.1038/12222 · Cited by 554 · cited 2×
“…and relational models have since become arguably the most pervasive framework for understanding hippocampal contributions to episodic memory (e.g., Davachi, 2006; Eichenbaum and Cohen, 2014; and Eichenbaum et al., 2007) and other forms of learning (Chun and Phelps, 1999; Goldfarb et al., 2016).”
BackgroundShaping of Object Representations in the Human Medial Temporal Lobe Based on Temporal Regularities
Schapiro, A. C., Kustner, L. V., & Turk-Browne, N. B. (2012) · Current Biology · doi:10.1016/j.cub.2012.06.056 · Cited by 430 · cited 1×
“In turn, learning statistical regularities in the co-occurrence of features can guide how features are integrated into perceptual units (Fiser and Aslin, 2002; Saffran et al., 1996) and neural representations (Schapiro et al., 2012, 2013).”
BackgroundSchönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
Self-citationContext, learning, and extinction
Gershman, S. J., Blei, D. M., & Niv, Y. (2010) · Psychological Review · doi:10.1037/a0017808 · Cited by 386 · cited 5×
“…predictive of weather outcomes. configurations and elements (Melchers et al., 2008) as well as computational frameworks that emphasize the use of structure in learning (e.g., Gershman et al., 2010), little is known about how the brain supports reinforcement learning when the cues are configural.”
BackgroundMethodInfluentialThe hippocampal-striatal axis in learning, prediction and goal-directed behavior
Pennartz, C. M. A., Ito, R., Verschure, P. F. M. J., Battaglia, F. P., & Robbins, T. W. (2011) · Trends in Neurosciences · doi:10.1016/j.tins.2011.08.001 · Cited by 287 · cited 1×
“This pathway provides a route through which configural associations in the hippocampus could be integrated with outcome signals in the nucleus accumbens (Ito et al., 2008; Pennartz et al., 2011).”
BackgroundTracking the emergence of conceptual knowledge during human decision making
Kumaran, D., Summerfield, J. J., Hassabis, D., & Maguire, E. A. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.030 · Cited by 260 · cited 2×
“…was first proposed by Sutherland and Rudy (1989) to account for the effects of isolated hippocampal lesions on instrumental and Pavlovian conditioning (Alvarado and Rudy, 1995; Dusek and Eichenbaum, 1998; Kumaran et al., 2009; but see Gallagher and Holland, 1992 and Saunders and Weiskrantz, 1989).”
The impact of study design on pattern estimation for single-trial multivariate pattern analysis
Mumford, J. A., Davis, T., & Poldrack, R. A. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.09.026 · Cited by 237
Persistence of hippocampal multivoxel patterns into postencoding rest is related to memory
Tambini, A., & Davachi, L. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1308499110 · Cited by 230
Role of the Basal Ganglia in Category Learning: How Do Patients With Parkinson's Disease Learn?
Shohamy, D., Myers, C. E., Onlaor, S., & Gluck, M. A. (2004) · Behavioral Neuroscience · doi:10.1037/0735-7044.118.4.676 · Cited by 186 · cited 3×
“Indeed, this observation has been quantified in prior work where individual differences in the use of different strategies were reported in a task that left the strategy options unconstrained (Gluck et al., 2002; Meeter et al., 2006; Shohamy et al., 2004).”
BackgroundMethodSelf-citationTop-down attention switches coupling between low-level and high-level areas of human visual cortex
Al-Aidroos, N., Said, C. P., & Turk-Browne, N. B. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1202095109 · Cited by 180
Cited by
Radulescu, A., Shin, Y. S., & Niv, Y. (2021) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-092920-120559 · Cited by 63
Parr, A. C., Calabro, F., Larsen, B., Tervo-Clemmens, B., Elliot, S., Foran, W., Olafsson, V., & Luna, B. (2021) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2021.101997 · Cited by 60
Mahmoodi, A., Luo, S., Harbison, C., Piray, P., & Rushworth, M. F. S. (2024) · Neuron · doi:10.1016/j.neuron.2024.09.001 · Cited by 17
Metadata
- Authors
- Duncan, K., Doll, B. B., Daw, N. D., & Shohamy, D.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2018-05
- Volume
- 98
- Issue
- 3
- Pages
- 645-657.e6
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W2800394030
- Cited by
- 75
- References
- 32
- FWCI
- 4.01
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.