Context, learning, and extinction
The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Abstract
A. Redish et al. (2007) proposed a reinforcement learning model of context-dependent learning and extinction in conditioning experiments, using the idea of "state classification" to categorize new observations into states. In the current article, the authors propose an interpretation of this idea in terms of normative statistical inference. They focus on renewal and latent inhibition, 2 conditioning paradigms in which contextual manipulations have been studied extensively, and show that online Bayesian inference within a model that assumes an unbounded number of latent causes can characterize a diverse set of behavioral results from such manipulations, some of which pose problems for the model of Redish et al. Moreover, in both paradigms, context dependence is absent in younger animals, or if hippocampal lesions are made prior to training. The authors suggest an explanation in terms of a restricted capacity to infer new causes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Large prediction errors signal that the current latent cause is no longer relevant (i.e., the situation has changed) and a new or stored latent cause should be inferred.”
From Does mental context drift or shift? (2017) · cites this paper as [43]
In the citing paper
“According to a recent theory of latent cause inference, people build up a library of statistical models of their environment, and then infer which is the hidden, or “latent,” cause of their current sensory observations [43–46].”
From Does mental context drift or shift? (2017) · cites this paper as [43]
In the citing paper
“From a statistical perspective, superposition of latent causes is straightforward to model; in fact, existing latent cause models infer a continuous probability distribution over latent causes [43].”
From Does mental context drift or shift? (2017) · cites this paper as [43]
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 9,459
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
(1978) · UA Campus Repository (The University of Arizona) · Cited by 8,010
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan, P., & Abbott, L. F. (2001) · Cited by 4,099
Simple memory: a theory for archicortex
Marr, D. (1971) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1971.0078 · Cited by 2,771 · cited 2×
“Our model may also shed new light on a long-standing question about the hippocampus and memory in general (Marr, 1971; McNaughton & Morris, 1987): When a new observation is made, under what circumstances is a new trace encoded or an old trace retrieved? Our model frames this as a choice between assigning an observatio…”
BackgroundMethod
Cited by
Items 101–200 of 379
Reinforcement learning with Marr
Niv, Y., & Langdon, A. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.04.005 · Cited by 64
Self-citationReinforcement Learning and Causal Models
Gershman, S. J. (2017) · Oxford Handbooks Online · doi:10.1093/oxfordhb/9780199399550.013.20 · Cited by 38
“The latent cause model, on the other hand, naturally explains latent inhibition in terms of changes in the joint probability of cues and outcomes (Gershman et al., 2010).”
BackgroundInfluentialSelf-citationBein, O., Gasser, C., Amer, T., Maril, A., & Davachi, L. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105368 · Cited by 64
Cocaine addiction as a homeostatic reinforcement learning disorder.
Keramati, M., Durand, A., Girardeau, P., Gutkin, B., & Ahmed, S. H. (2017) · Psychological Review · doi:10.1037/rev0000046 · Cited by 44
Compositional clustering in task structure learning
Franklin, N., & Frank, M. J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006116 · Cited by 59
“This logic 131 underlies models of animal Pavlovian learning and transfer [2], human instrumental learning and 132 transfer [3, 4], and category learning [14].”
BackgroundUnraveling the Mysteries of Motivation
O’Reilly, R. C. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.03.001 · Cited by 57
“Computational models of these systems at multiple levels of analysis show how these interacting systems can produce the signature phenomena of classical and instrumental conditioning, thereby explaining how perceived rewards and punishments can modulate behavior [52, 53, 54, 55].”
BackgroundExtinction: Behavioral Mechanisms and Their Implications
Bouton, M. E., & Woods, A. M. (2008) · Elsevier eBooks · doi:10.1016/b978-012370509-9.00088-7 · Cited by 52
Perceptual estimation obeys Occam's razor
Gershman, S. J., & Niv, Y. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00623 · Cited by 45
Self-citationTrait anxiety is associated with hidden state inference during aversive reversal learning
Zika, O., Wiech, K., Reinecke, A., Browning, M. D., & Schuck, N. W. (2023) · Nature Communications · doi:10.1038/s41467-023-39825-3 · Cited by 54
“We note that a number of state switching models have been proposed previously 69,70,83,84 but none of these meets the goals set above.”
Computational processes of simultaneous learning of stochasticity and volatility in humans
Piray, P., & Daw, N. D. (2024) · Nature Communications · doi:10.1038/s41467-024-53459-z · Cited by 45
“These additional issues include attention 2 , decision noise and the explore-exploit tradeoff 47,59 , social cause inference 27,60,61 , the organization of experiences into latent causes or contexts 5,62,63 , and planning 64 – 67 .”
Baudonnat, M., Huber, A., David, V., & Walton, M. E. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00175 · Cited by 44
“For instance, as suggested by Gershman, Niv and colleagues in a series of papers (Gershman et al., 2010; Gershman and Niv, 2010), the magnitude of an RPE signaled by dopamine release might be used to indicate whether or not an environment has www.frontiersin.org October 2013 | Volume 7 | Article…”
Elucidating Poor Decision-Making in a Rat Gambling Task
Rivalan, M., Valton, V., Seriès, P., Marchand, A. R., & Dellu-Hagedorn, F. (2013) · PLoS ONE · doi:10.1371/journal.pone.0082052 · Cited by 45
“Other formalisms such as win-stay looseshift, Bayesian models or more elaborate TD models could also be explored [18,19,20,22,23,24].”
BackgroundAdvanced Reinforcement Learning
Daw, N. D. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00016-4 · Cited by 54
“Behaviorally, learning and choice behavior in another RL task with a hidden state is well explained as resulting from such a hybrid scheme of combining Bayesian inference of the latent state with TD for learning action values over the inferred state (Gershman et al., 2010b).”
BackgroundImprecise neural computations as a source of adaptive behaviour in volatile environments
Findling, C., Chopin, N., & Koechlin, É. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-00971-z · Cited by 61
Gershman, S. J., & Cikara, M. (2020) · Current Directions in Psychological Science · doi:10.1177/0963721420924481 · Cited by 50
Self-citationIigaya, K. (2016) · eLife · doi:10.7554/elife.18073 · Cited by 43
“Second, although we assumed that the surprise signal would reset most of the accumulated evidence when reward-harvesting performance deterio-rates, in many cases it would be better to keep accumulated evidence, such as to form distinct ’con-texts’ ( Gershman et al., 2010 ; Lloyd and Leslie, 2013 ).”
BackgroundContext-sensitive valuation and learning
Hunter, L. E., & Daw, N. D. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.05.001 · Cited by 45
The Development of Executive Function: Mechanisms of Change and Functional Pressures
Ibbotson, P. (2023) · Journal of Cognition and Development · doi:10.1080/15248372.2022.2160719 · Cited by 59
“…abstraction over network depth as a fundamental organizational feature of the brain (Felleman & Van Essen, 1991; Taylor et al., 2015) cognition and behavior (Anderson, 1991, Sanborn et al., 2010; Griffiths & Austerweil, 2008; Kemp et al., 2010; Gershman et al., 2010; Goldwater et al., 2009).”
BackgroundFear and safety learning in anxiety- and stress-related disorders: An updated meta-analysis
Kausche, F. M., Carsten, H. P., Sobania, K. M., & Riesel, A. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105983 · Cited by 63
Hanson, J. L., Gillmore, A. D., Yu, T., Holmes, C. J., Hallowell, E. S., Barton, A. W., Beach, S. R. H., Galván, A., MacKillop, J., Windle, M., et al. (2019) · Child Development · doi:10.1111/cdev.13154 · Cited by 41
Context-dependent decision-making in the primate hippocampal–prefrontal circuit
Elston, T. W., & Wallis, J. D. (2025) · Nature Neuroscience · doi:10.1038/s41593-024-01839-5 · Cited by 45
“Their aim is to create a state transition graph that describes a behavioral task with a minimum number of necessary states 35,36 .”
Culver, N. C., Stevens, S., Fanselow, M. S., & Craske, M. G. (2017) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2017.07.003 · Cited by 44
“According to Gershman, Blei, and Niv (2010), such large prediction errors cause a new CS-no US memory to be formed in a novel state (i.e., context) that is specific to extinction.”
BackgroundOverharvesting in human patch foraging reflects rational structure learning and adaptive planning
Harhen, N. C., & Bornstein, A. M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2216524120 · Cited by 61
Fear of the unexpected: Hippocampus mediates novelty-induced return of extinguished fear in rats
Maren, S. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2013.06.004 · Cited by 38
“This approach has had some success in explaining the context-dependence of extinction (Gershman et al., 2010).”
Navigating complex decision spaces: Problems and paradigms in sequential choice.
Walsh, M. M., & Anderson, J. R. (2014) · Psychological Bulletin · doi:10.1037/a0033455 · Cited by 38
“How humans and animals form such a representation remains an active research topic (Gershman et al., 2010; Kemp & Tenenbaum, 2008).”
BackgroundA flexible and generalizable model of online latent-state learning
Cochran, A. L., & Cisler, J. M. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007331 · Cited by 40
“Many latent-state learning models assume the number of latent state can grow [11, 15, 17].”
InfluentialKritman, M., & Maroun, M. (2013) · International Journal of Neuropsychopharmacology · doi:10.1017/s1461145712000636 · Cited by 38
Multiple stages of learning in perceptual categorization: evidence and neurocomputational theory
Cantwell, G. T., Crossley, M. J., & Ashby, F. G. (2015) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0827-2 · Cited by 46
“…the S-R type, a number assume that the S-R learning process is mediated via the interaction of multiple systems (Ashby et al., 1998; Erickson & Kruschke, 1998), or that SR associations are formed via context-dependent Bayesian inference (Anderson, 1991; Gershman et al., 2010; Redish et al., 2007).”
InfluentialThe Consumer Contextual Decision-Making Model
Suomala, J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.570430 · Cited by 57
A Formal Valuation Framework for Emotions and Their Control
Huys, Q. J. M., & Renz, D. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.07.003 · Cited by 45
“This has been applied to a wide variety of affective responses, including heart rate changes (45), approach (46), avoidance (47,48), extinction (49), vigor (50,51), and others.”
BackgroundEvidence for hippocampus-dependent contextual learning at postnatal day 17 in the rat
Foster, J. A., & Burman, M. A. (2010) · Learning & Memory · doi:10.1101/lm.1755810 · Cited by 45
“These data are perhaps best interpreted as contributing to a growing literature suggesting that rats around PD 16–18 are able to encode contextual information, but are unable to act on that information until further development has occurred (Yap and Richardson 2005, 2007; Gershman et al. 2010).”
BackgroundResultLinking dynamic patterns of neural activity in orbitofrontal cortex with decision making
Rich, E. L., Stoll, F. M., & Rudebeck, P. H. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.11.002 · Cited by 40
Learning and transfer of working memory gating policies
Bhandari, A., & Badre, D. (2017) · Cognition · doi:10.1016/j.cognition.2017.12.001 · Cited by 43
Adaptive Learning through Temporal Dynamics of State Representation
Razmi, N., & Nassar, M. R. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.0387-21.2022 · Cited by 38
State representation in mental illness
Radulescu, A., & Niv, Y. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.03.011 · Cited by 46
“All else equal, we should strive to explain current observations using the latent causes we already know exist, rather than assume new ones [59].”
BackgroundSelf-citationLearning to Synchronize: Midfrontal Theta Dynamics during Rule Switching
Verbeke, P., Ergo, K., Loof, E. D., & Verguts, T. (2020) · Journal of Neuroscience · doi:10.1523/jneurosci.1874-20.2020 · Cited by 40
The partial-reinforcement extinction effect and the contingent-sampling hypothesis
Hochman, G., & Erev, I. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0432-1 · Cited by 32
Children Use Temporal Cues to Learn Causal Directionality
Rottman, B. M., Kominsky, J. F., & Keil, F. C. (2013) · Cognitive Science · doi:10.1111/cogs.12070 · Cited by 30
Mawase, F., Bar‐Haim, S., Joubran, K., Rubin, L., Karniel, A., & Shmuelof, L. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00203 · Cited by 34
“Frontiers in Human Neuroscience | www.frontiersin.org 5 May 2016 | Volume 10 | Article 203 FIGURE 3 | Changes in variability in adaptation and washout sessions.”
BackgroundInfluentialEvaluation of ambiguous associations in the amygdala by learning the structure of the environment
Madarász, T. J., Díaz-Mataix, L., Akhand, O., Ycu, E. A., LeDoux, J. E., & Johansen, J. P. (2016) · Nature Neuroscience · doi:10.1038/nn.4308 · Cited by 32
Bein, O., Trzewik, M., & Maril, A. (2019) · Journal of Memory and Language · doi:10.1016/j.jml.2019.03.006 · Cited by 29
Franklin, N. T., & Frank, M. J. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007720 · Cited by 47
“Previous models have considered how agents and animals can cluster “latent states” across multiple contexts that share task statistics in both Pavlovian [2] and instrumental learning settings [3, 4].”
BackgroundA generative framework for the study of delusions
Erdmann, T., & Mathys, C. D. (2021) · Schizophrenia Research · doi:10.1016/j.schres.2020.11.048 · Cited by 38
“Gershman et al. (2010) showed how state classi fi cation can be derived as rational inference in a Dirichlet process mixture model.”
Balancing control: A Bayesian interpretation of habitual and goal-directed behavior
Schwöbel, S., Marković, D., Smolka, M. N., & Kiebel, S. J. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102472 · Cited by 37
An engram of intentionally forgotten information
ten Oever, S., Sack, A. T., Oehrn, C. R., & Axmacher, N. (2021) · Nature Communications · doi:10.1038/s41467-021-26713-x · Cited by 28
Kupferschmidt, D. A., Cummings, K. A., Joffe, M. E., MacAskill, A. F., Malik, R., Sánchez-Bellot, C., Tejeda, H. A., & Yarur, H. E. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.1136-22.2022 · Cited by 30
“…and updating of affective associations (e.g., aversive threat memory) (Sotres-Bayon et al., 2012; Janak and Tye, 2015), whereas vHPC inputs are thought to convey context, or state information on which associations can be formed (Gershman et al., 2010; Maren et al., 2013; Marek et al., 2018).”
BackgroundErasing the engram: The unlearning of procedural skills.
Crossley, M. J., Ashby, F. G., & Maddox, W. T. (2013) · Journal of Experimental Psychology General · doi:10.1037/a0030059 · Cited by 30
“Also, the models are not neurobiologically detailed, although Redish et al. (2007) and Gershman et al. (2010) both speculated that the locus of their context-learning module is within prefrontal cortex and/or the hippocampus.”
Learning bundles of stimuli renders stimulus order as a cue, not a confound
Qian, T., & Aslin, R. Ν. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1416109111 · Cited by 26
Structure and Flexibility in Bayesian Models of Cognition
Austerweil, J. L., Gershman, S. J., & Griffiths, T. L. (2015) · Oxford Handbooks Online · doi:10.1093/oxfordhb/9780199957996.013.9 · Cited by 27
“…Murphy, 1999; Pothos and Chater, 2002), motion perception (Braddick, 1993), causal inference (Kemp, Tenenbaum, Niyogi, & Griffiths, 2010), word segmentation (Werker and Yeung, 2005), semantic representation (Griffiths, Steyvers, & Tenenbaum, 2007), and associative learning (Gershman et al., 2010).”
BackgroundMethodSelf-citationMixing memory and desire: How memory reactivation supports deliberative decision‐making
Wang, S., Feng, S. F., & Bornstein, A. M. (2021) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1581 · Cited by 30
Norbury, A., Brinkman, H. R., Kowalchyk, M. L., Monti, E., Pietrzak, R. H., Schiller, D., & Feder, A. (2021) · Psychological Medicine · doi:10.1017/s0033291721000647 · Cited by 28
“, if contextual cue changes mean that the old fear memory is retrieved, rather than the new extinction memory) (Gershman et al., 2010; Gershman & Niv, 2012; Gershman, Norman, & Niv, 2015).”
BackgroundContributions of the basal ganglia to action sequence learning and performance
Garr, E. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.09.017 · Cited by 32
Extinction: Behavioral Mechanisms and Their Implications ☆
Bouton, M. E. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-809324-5.21006-7 · Cited by 31
Multi-Hop Knowledge Graph Reasoning in Few-Shot Scenarios
Zheng, S., Chen, W., Wang, W., Zhao, P., Yin, H., & Zhao, L. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3304665 · Cited by 30
Sunsay, C., & Rebec, G. V. (2014) · Behavioral Neuroscience · doi:10.1037/bne0000012 · Cited by 29
“Presentation of non-contingent rewards helps the animal infer that acquisition is active at test (Gershman et al., 2010).”
BackgroundBiased evaluations emerge from inferring hidden causes
Shin, Y. S., & Niv, Y. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01065-0 · Cited by 30
Self-citationA systems-neuroscience model of phasic dopamine.
Mollick, J., Hazy, T. E., Krueger, K. A., Nair, A., Mackie, P., Herd, S. A., & O’Reilly, R. C. (2020) · Psychological Review · doi:10.1037/rev0000199 · Cited by 27
Varga, N. L., Morton, N. W., & Preston, A. R. (2024) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780190917982.013.47 · Cited by 30
Exposure Techniques: The Role of Extinction Learning
Urcelay, G. P. (2012) · Springer eBooks · doi:10.1007/978-1-4614-3342-2_4 · Cited by 26
Latent-state and model-based learning in PTSD
Cisler, J. M., Dunsmoor, J. E., Fonzo, G. A., & Nemeroff, C. B. (2024) · Trends in Neurosciences · doi:10.1016/j.tins.2023.12.002 · Cited by 26
Zhu, J.-Q., Sundh, J., Spicer, J., Chater, N., & Sanborn, A. N. (2024) · Psychological Review · doi:10.1037/rev0000427 · Cited by 48
“…explaining effects in other areas of psychologysuch as vision (e.g., Yuille &Kersten,2006),motorcontrol (K ö rding & Wolpert, 2004), causal reasoning (e.g., Abbott & Grif fi ths, 2011; Bramley et al., 2017), reading (Norris, 2006), and learning (e.g., Courville & Daw, 2007; Gershman et al., 2010).”
BackgroundAn associability decay model of paradoxical choice.
Daniels, C. W., & Sanabria, F. (2018) · Journal of Experimental Psychology Animal Learning and Cognition · doi:10.1037/xan0000179 · Cited by 21
The state of the orbitofrontal cortex
Sharpe, M. J., Wikenheiser, A. M., Niv, Y., & Schoenbaum, G. (2015) · Neuron · doi:10.1016/j.neuron.2015.12.004 · Cited by 21
“Yet an appreciation of the contribution of prediction errors to segregating learning between different inferred states (Gershman et al., 2010) indicates that for extinction to be effective it must be conducted under circumstances—a state—similar to the original learning, otherwise the relevant…”
BackgroundSelf-citationAssessment IS learning: developing a student‐centred approach for assessment in Higher Education
Rutherford, S., Pritchard, C., & Francis, N. (2024) · FEBS Open Bio · doi:10.1002/2211-5463.13921 · Cited by 51
Incremental implicit learning of bundles of statistical patterns
Qian, T., Jaeger, T. F., & Aslin, R. Ν. (2016) · Cognition · doi:10.1016/j.cognition.2016.09.002 · Cited by 25
“One direction is to explore computational methods that approximate the inferences assumed by the ideal learner, but are computationally simpler and cognitively more plausible (Gershman et al., 2010; Griffiths, Vul, & Sanborn, 2012; Lloyd & Leslie, 2013; Sanborn et al., 2010).”
BackgroundDezfouli, A., & Balleine, B. W. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007334 · Cited by 24
“Recent theoretical work provides a basis for this process in the context of classical conditioning and suggests that the underlying states used to represent the environment are dynamic; that animals are able to infer and learn new states of the environment based on their observations [3, 4].”
BackgroundAssociative Learning from Replayed ExperiencePreprint
Ludvig, E. A., Mirian, M. S., Kehoe, E. J., & Sutton, R. S. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/100800 · Cited by 23
Learning and Generalization under Ambiguity: An fMRI Study
Chumbley, J. R., Flandin, G., Bach, D. R., Daunizeau, J., Fehr, E., Dolan, R. J., & Friston, K. J. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002346 · Cited by 20
“It is also related to a previous [42] Bayesian formulation of”
BackgroundMethodInfluentialDias, C., Wang, Y. T., & Phillips, A. G. (2012) · Behavioural Brain Research · doi:10.1016/j.bbr.2012.05.026 · Cited by 22
The Computational Nature of Memory ModificationPreprint
Gershman, S. J., Monfils, M., Norman, K. A., & Niv, Y. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/036442 · Cited by 21
Self-citationTemporal prediction errors modulate task-switching performance
Limongi, R., Silva, A. M., & Góngora-Costa, B. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01185 · Cited by 19
“However, Bayesian models of associative learning also include PEs as a driving variable (Kruschke, 2008; Gershman et al., 2010).”
BackgroundAcute inhibition of mGluR5 disrupts behavioral flexibility
Sethna, F., & Wang, H. (2016) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2016.01.004 · Cited by 21
Orbitofrontal cortex and learning predictions of state transitions.
Chan, S. C. Y., Schuck, N. W., Lopatina, N., Schoenbaum, G. M., & Niv, Y. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000461 · Cited by 23
Self-citationBlocked training facilitates learning of multiple schemas
Beukers, A. O., Collin, S. H. P., Kempner, R. P., Franklin, N. T., Gershman, S. J., & Norman, K. A. (2024) · Communications Psychology · doi:10.1038/s44271-024-00079-4 · Cited by 29
Self-citationMinimum description length model selection in associative learning
Gallistel, C. R., & Wilkes, J. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.02.025 · Cited by 18
Uncertainty-Dependent Extinction of Fear Memory in an Amygdala-mPFC Neural Circuit Model
Li, Y., Nakae, K., Ishii, S., & Naoki, H. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005099 · Cited by 23
“[99] developed a Bayesian inference model based on a categorization process of contexts [9].”
MethodOrbitofrontal State Representations Are Related to Choice Adaptations and Reward Predictions
Stalnaker, T. A., Raheja, N., & Schoenbaum, G. M. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.0753-20.2020 · Cited by 23
“In this case, the map would represent that in one state, when a correct left response yields three drops of chocolate milk, portant to distinguish whether a new state has been entered or the old state is changing (Gershman et al., 2010; Niv, 2019).”
BackgroundInfluentialMahmoodi, A., Luo, S., Harbison, C., Piray, P., & Rushworth, M. F. S. (2024) · Neuron · doi:10.1016/j.neuron.2024.09.001 · Cited by 17
Pettine, W. W., Raman, D. V., Redish, A. D., & Murray, J. D. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01543-7 · Cited by 25
“The tutorial was implemented with this structure for three reasons: (1) to instruct participants on task structure; (2) to screen out bots; and (3) to train participants to expect stochastic rewards, as well as a multi-way mapping between actions and states.”
BackgroundMethodInfluentialHow Did the Chicken Cross the Road? With Her Striatal Cholinergic Interneurons, Of Course
Schoenbaum, G. M., Stalnaker, T. A., & Niv, Y. (2013) · Neuron · doi:10.1016/j.neuron.2013.06.033 · Cited by 19
“Reinstatement indicates that extinction training did not simply overwrite and erase previous associations between actions and outcomes (Gershman et al., 2010), but rather reward omission caused rats in both groups to create a new state (Figure 1, state 4).”
BackgroundSelf-citationBai, J. Y. H., & Podlesnik, C. A. (2017) · Behavioural Processes · doi:10.1016/j.beproc.2017.02.011 · Cited by 17
Mohanta, S., Afrasiabi, M., Casey, C., Tanabe, S., Redinbaugh, M. J., Kambi, N. A., Phillips, J. M., Polyakov, D., Filbey, W., Austerweil, J. L., et al. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.1311-21.2021 · Cited by 20
Auchter, A., Cormack, L. K., Niv, Y., Gonzalez-Lima, F., & Monfils, M. (2017) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2017.00002 · Cited by 21
“New normative learning models posit that during extinction training, animals may infer a new state that is different from that in acquisition training, and thereby form (and update) a new association or memory, rather than apply the experiences during extinction to updating the old acquisition association (Redish et a…”
BackgroundInfluentialSelf-citationA double error dynamic asymptote model of associative learning.
Kokkola, N. H., Mondragón, E., & Alonso, E. (2019) · Psychological Review · doi:10.1037/rev0000147 · Cited by 23
Using computational models of learning to advance cognitive behavioral therapy
Berwian, I. M., Hitchock, P., Pisupati, S., Schoen, G., & Niv, Y. (2025) · Communications Psychology · doi:10.1038/s44271-025-00251-4 · Cited by 16
Self-citationJain, Y., Callaway, F., Griffiths, T. L., Dayan, P., He, R., Krueger, P. M., & Lieder, F. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01789-5 · Cited by 26
“, 2018c), others suggest that animals change their strategy abruptly when they detect a change in the environment (Gershman et al., 2010).”
BackgroundRoyer, M., Pai, B., Menon, R., Bludau, A., Gryksa, K., Perry, R. B., Ulitsky, I., Meister, G., & Neumann, I. D. (2022) · Molecular Psychiatry · doi:10.1038/s41380-022-01481-2 · Cited by 20
Computational approaches to habits in a model-free world
Pauli, W. M., Cockburn, J., Pool, E. R., Perez, O. D., & O’Doherty, J. P. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.12.001 · Cited by 20
“These models posit that internal goals, and beliefs about hidden states of the environment, influence the interpretation of the behavioral context [34,35].”
BackgroundLearning Structures Through Reinforcement
Collins, A. G. E. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812098-9.00005-x · Cited by 17
Liu, R. G., & Frank, M. J. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103770 · Cited by 19
Predictive Representations: Building Blocks of Intelligence
Carvalho, W., Tomov, M. S., de Cothi, W., Barry, C., & Gershman, S. J. (2024) · Neural Computation · doi:10.1162/neco_a_01705 · Cited by 15
Self-citationExploration from Generalization Mediated by Multiple Controllers
Dayan, P. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_4 · Cited by 16
“From the perspective of model-based RL, the subjects may be building a causal description of the original environment (Courville et al. 2004; Gershman et al. 2010b; Tenenbaum et al. 2006) indicating what could and could not be responsible for their dismal fate.”
BackgroundComputational Mechanisms of Addiction and Anxiety: A Developmental Perspective
Goldway, N., Eldar, E., Shoval, G., & Hartley, C. A. (2023) · Biological Psychiatry · doi:10.1016/j.biopsych.2023.02.004 · Cited by 18
“Under this computational account, learners segment their experience into latent states that capture regularities in the con fi guration of observed stimuli (e.g., CS and US) (111).”
BackgroundNatural forgetting reversibly modulates engram expressionPreprint
O’Leary, J. D., Bruckner, R., Autore, L., & Ryan, T. J. (2024) · eLife · doi:10.7554/elife.92860 · Cited by 25
“Several approaches to modeling extinction ( Redish et al., 2007 ; Gershman et al., 2010 ; Maren et al., 2013 ), memory interference ( Gershman et al., 2014 ), or the creation and updating of motor memories ( Heald et al., 2021 ) that more explicitly assumed multiple memory representations could…”
MethodEmergence of complex dynamics of choice due to repeated exposures to extinction learning
Donoso, J. R., Packheiser, J., Pusch, R., Lederer, Z., Walther, T., Uengoer, M., Lachnit, H., Güntürkün, O., & Cheng, S. (2021) · Animal Cognition · doi:10.1007/s10071-021-01521-4 · Cited by 18
“, the intermittence of the renewal effect) might have been concealed by averaging in the previous studies (André et al. 2015; Cheng and Sabes 2006, 2007; Gershman et al. 2010; Larrauri and Schmajuk 2008; Lissek et al. 2015; Todd 2013; Trask et al. 2017; Wirth et al. 2003).”
BackgroundResultInfluentialStayte, S., Dhungana, A., Vissel, B., & Bradfield, L. A. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2021.655029 · Cited by 16
“That is, as shown in Figure 1 and in a manner reminiscent of Yael Niv’s latent cause theory (Gershman et al., 2010; Gershman and Niv, 2012); if the animal infers the initial lever press acquisition state/latent cause [state 1 (S1)] it will press the lever to earn a pellet, but if it infers the degradation state/latent…”
MethodThe orbitofrontal cortex updates beliefs for state inference
Schiereck, S. S., Pérez-Rivera, D. T., Mah, A., DeMaegd, M. L., Hocker, D. L., Ward, R. M., Savin, C., & Constantinople, C. M. (2025) · Neuron · doi:10.1016/j.neuron.2025.10.024 · Cited by 16
Mondragón, E., Gray, J., Alonso, E., Bonardi, C., & Jennings, D. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0102469 · Cited by 12
“Following standard practice in the field (e.g., [53–56]) simulated results were compared against published experimental data by visual inspection of their respective CR patterns.”
Zhang, L., Chen, X., Sindreu, C. B., Song, L., Storm, D. R., Zweifel, L. S., & Xia, Z. (2019) · PLoS ONE · doi:10.1371/journal.pone.0219152 · Cited by 15
Protection from uncertainty in the exploration/exploitation trade-off.
Walker, A. R., Navarro, D., Newell, B. R., & Beesley, T. (2021) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000883 · Cited by 19
Metadata
- Authors
- Gershman, S. J., Blei, D. M., & Niv, Y.
- Affiliations
- Princeton University (US)
- Venue
- Psychological Review
- Published
- 2010
- Volume
- 117
- Issue
- 1
- Pages
- 197-209
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- OpenAlex
- W2086710210
- Cited by
- 386
- References
- 74
- Citations by year
- 2026: 26 · 2025: 29 · 2024: 38 · 2023: 27 · 2022: 24 · 2021: 33 · 2020: 22 · 2019: 32 · 2018: 21 · 2017: 26 · 2016: 29 · 2015: 17 · 2014: 15 · 2013: 16 · 2012: 15
- FWCI
- 8.79
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Review (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Google and Microsoft Research
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- context-dependent learning, extinction, reinforcement learning, state classification, Bayesian inference, latent inhibition, renewal, hippocampus, age differences
- MeSH terms
- Bayes Theorem, Conditioning, Classical, Extinction, Psychological, Hippocampus, Humans, Learning, Models, Psychological, Semantics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.