Context, learning, and extinction
The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Abstract
A. Redish et al. (2007) proposed a reinforcement learning model of context-dependent learning and extinction in conditioning experiments, using the idea of "state classification" to categorize new observations into states. In the current article, the authors propose an interpretation of this idea in terms of normative statistical inference. They focus on renewal and latent inhibition, 2 conditioning paradigms in which contextual manipulations have been studied extensively, and show that online Bayesian inference within a model that assumes an unbounded number of latent causes can characterize a diverse set of behavioral results from such manipulations, some of which pose problems for the model of Redish et al. Moreover, in both paradigms, context dependence is absent in younger animals, or if hippocampal lesions are made prior to training. The authors suggest an explanation in terms of a restricted capacity to infer new causes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Large prediction errors signal that the current latent cause is no longer relevant (i.e., the situation has changed) and a new or stored latent cause should be inferred.”
From Does mental context drift or shift? (2017) · cites this paper as [43]
In the citing paper
“According to a recent theory of latent cause inference, people build up a library of statistical models of their environment, and then infer which is the hidden, or “latent,” cause of their current sensory observations [43–46].”
From Does mental context drift or shift? (2017) · cites this paper as [43]
In the citing paper
“From a statistical perspective, superposition of latent causes is straightforward to model; in fact, existing latent cause models infer a continuous probability distribution over latent causes [43].”
From Does mental context drift or shift? (2017) · cites this paper as [43]
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 9,461
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
(1978) · UA Campus Repository (The University of Arizona) · Cited by 8,010
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan, P., & Abbott, L. F. (2001) · Cited by 4,099
Simple memory: a theory for archicortex
Marr, D. (1971) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1971.0078 · Cited by 2,771 · cited 2×
“Our model may also shed new light on a long-standing question about the hippocampus and memory in general (Marr, 1971; McNaughton & Morris, 1987): When a new observation is made, under what circumstances is a new trace encoded or an old trace retrieved? Our model frames this as a choice between assigning an observatio…”
BackgroundMethodThe hippocampal-VTA loop: controlling the entry of information into long-term memory
Lisman, J. E., & Grace, A. A. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.002 · Cited by 1,983
Grossberg, S. (1976) · Biological Cybernetics · doi:10.1007/bf00344744 · Cited by 1,909
Context, time, and memory retrieval in the interference paradigms of Pavlovian learning
Bouton, M. E. (1993) · Psychological Bulletin · doi:10.1037/0033-2909.114.1.80 · Cited by 1,896
Context and Behavioral Processes in Extinction: Table 1.
Bouton, M. E. (2004) · Learning & Memory · doi:10.1101/lm.78804 · Cited by 1,808 · cited 2×
“One motivation for such models is the finding that learned relationships between cues and reinforcement are not necessarily erased following extinction: Returning the animal to the original training context after extinction in a different context can lead to renewal of the conditioned response (Bouton, 2004; Bouton &…”
BackgroundResultExchangeability and related topics
Aldous, D. J. (1985) · Lecture Notes in Mathematics · doi:10.1007/bfb0099421 · Cited by 1,345
Hippocampal synaptic enhancement and information storage within a distributed memory system
McNaughton, B. L., & Morris, R. G. M. (1987) · Trends in Neurosciences · doi:10.1016/0166-2236(87)90011-7 · Cited by 1,304
On the role of the hippocampus in learning and memory in the rat.
Jarrard, L. E. (1993) · Behavioral and Neural Biology · doi:10.1016/0163-1047(93)90664-4 · Cited by 1,125
Aguado Aguilar, L. (2001) · PubMed · Cited by 1,072
The adaptive nature of human categorization.
Anderson, J. R. (1991) · Psychological Review · doi:10.1037/0033-295x.98.3.409 · Cited by 1,069
The hippocampus and contextual retrieval of information from memory: A theory
Hirsh, R. (1974) · Behavioral Biology · doi:10.1016/s0091-6773(74)92231-7 · Cited by 957
Combinatorial Stochastic Processes
(2006) · Lecture Notes in Mathematics · doi:10.1007/b11601500 · Cited by 935
Hippocampal conjunctive encoding, storage, and recall: Avoiding a trade‐off
O'Reilly, R. C., & McClelland, J. L. (1994) · Hippocampus · doi:10.1002/hipo.450040605 · Cited by 914
Hasselmo, M. E., Bodelón, C., & Wyble, B. P. (2002) · Neural Computation · doi:10.1162/089976602317318965 · Cited by 875
Contextual control of the extinction of conditioned fear
Bouton, M. E., & Bolles, R. C. (1979) · Learning and Motivation · doi:10.1016/0023-9690(79)90057-2 · Cited by 791
Bouton, M. E., & King, D. A. (1983) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.9.3.248 · Cited by 769
Prefrontal mechanisms in extinction of conditioned fear
Quirk, G. J., Garcia, R., & González-Lima, F. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.03.010 · Cited by 682
The discovery of structural form
Kemp, C., & Tenenbaum, J. B. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0802631105 · Cited by 650
Hippocampal mediation of stimulus representation: A computational theory
Gluck, M. A., & Myers, C. E. (1993) · Hippocampus · doi:10.1002/hipo.450030410 · Cited by 501
Learning and selective attention
Dayan, P., Kakade, S., & Montague, P. R. (2000) · Nature Neuroscience · doi:10.1038/81504 · Cited by 495 · cited 1×
“An exception is the competitive mixture of experts model of Dayan and colleagues (Dayan & Long, 1998; Dayan et al., 2000), in which reinforcement is assumed to be generated by a single observable cause.”
BackgroundTheories of Associative Learning in Animals
Pearce, J. M., & Bouton, M. E. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.111 · Cited by 492
Brogden, W. J. (1939) · Journal of Experimental Psychology · doi:10.1037/h0058944 · Cited by 409
The misbehavior of value and the discipline of the will
Dayan, P., Niv, Y., Seymour, B., & D. Daw, N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.002 · Cited by 388
Self-citationBeyond the Cognitive Map: From Place Cells to Episodic Memory
Redish, A. D. (1999) · Cited by 370
Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346 · cited 5×
“To this end, we propose a new model that is conceptually aligned with that of Redish et al. (2007) but more directly descended from the latent cause theory of Courville (2006).”
BackgroundMethodResultInfluentialHoney, R. C., & Good, M. (1993) · Behavioral Neuroscience · doi:10.1037//0735-7044.107.1.23 · Cited by 336
Hippocampal mechanisms for the context-dependent retrieval of episodes
Hasselmo, M. E., & Eichenbaum, H. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.08.007 · Cited by 298
Hippocampal involvement in contextual modulation of fear extinction
Ji, J., & Maren, S. (2007) · Hippocampus · doi:10.1002/hipo.20331 · Cited by 297
A model of hippocampally dependent navigation, using the temporal difference learning rule
Foster, D. J., Morris, R. G. M., & Dayan, P. (2000) · Hippocampus · doi:10.1002/(sici)1098-1063(2000)10:1<1::aid-hipo1>3.0.co;2-1 · Cited by 290
Blaisdell, A. P., Sawa, K., Leising, K. J., & Waldmann, M. R. (2006) · Science · doi:10.1126/science.1121872 · Cited by 287
Howard, M. W., Fotedar, M. S., Datey, A. V., & Hasselmo, M. E. (2005) · Psychological Review · doi:10.1037/0033-295x.112.1.75 · Cited by 283
Contextual fear conditioning, conjunctive representations, pattern completion, and the hippocampus.
Rudy, J. W., & O'Reilly, R. C. (1999) · Behavioral Neuroscience · doi:10.1037//0735-7044.113.5.867 · Cited by 281
Integrating hippocampus and striatum in decision-making
Johnson, A., van der Meer, M. A., & Redish, A. D. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.01.003 · Cited by 253
Hall, G., & Honey, R. C. (1989) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037//0097-7403.15.3.232 · Cited by 240
Hippocampus: cognitive map or working memory?
Nadel, L., & MacDonald, L. (1980) · Behavioral and Neural Biology · doi:10.1016/s0163-1047(80)90430-6 · Cited by 219
Contextual conditioning and auditory cue conditioning dissociate during development.
Rudy, J. W. (1993) · Behavioral Neuroscience · doi:10.1037//0735-7044.107.5.887 · Cited by 218
Representation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173 · cited 1×
“In the Pavlovian version of the renewal paradigm described here, the animal's conditioned response is presumed to be proportional to the current state's value (see Dayan, Niv, Seymour, & Daw, 2006).”
BackgroundElectrolytic lesions of the dorsal hippocampus disrupt renewal of conditional fear after extinction
Ji, J., & Maren, S. (2005) · Learning & Memory · doi:10.1101/lm.91705 · Cited by 160
The Role of the Hippocampus in Instrumental Conditioning
Corbit, L. H., & Balleine, B. W. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-11-04233.2000 · Cited by 145
The role of the rat hippocampal system in several effects of context in extinction.
Wilson, A., Brooks, D. C., & Bouton, M. E. (1995) · Behavioral Neuroscience · doi:10.1037//0735-7044.109.5.828 · Cited by 142
Proceedings of the 28th Annual Conference of the Cognitive Science Society
Sun, R. (2006) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 142
Brandon, S. E., Vogel, E. H., & Wagner, A. R. (2000) · Behavioural Brain Research · doi:10.1016/s0166-4328(99)00185-0 · Cited by 141
The effects of neurotoxic hippocampal lesions on two effects of context after fear extinction.
Frohardt, R. J., Guarraci, F. A., & Bouton, M. E. (2000) · Behavioral Neuroscience · doi:10.1037//0735-7044.114.2.227 · Cited by 139
The dynamic hierarchical Dirichlet process
Ren, L., Dunson, D. B., & Carin, L. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390260 · Cited by 131
Detecting and predicting changes
Brown, S. D., & Mark Steyvers, B. (2008) · Cognitive Psychology · doi:10.1016/j.cogpsych.2008.09.002 · Cited by 129
Particle filters for mixture models with an unknown number of components
Fearnhead, P. (2003) · Statistics and Computing · doi:10.1023/b:stco.0000009418.04621.cd · Cited by 107
A more rational model of categorization
Sanborn, A. N., Griffiths, T. L., & Navarro, D. (2006) · Warwick Research Archive Portal (University of Warwick) · doi:10.82444/warw.33600547 · Cited by 103
Acquisition and extinction in autoshaping.
Kakade, S., & Dayan, P. (2002) · Psychological Review · doi:10.1037//0033-295x.109.3.533 · Cited by 102
Love, B. C., & Gureckis, T. M. (2007) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.7.2.90 · Cited by 102
Lesions of rat infralimbic cortex enhance renewal of extinguished appetitive Pavlovian responding
Rhodes, S. E. V., & Killcross, A. S. (2007) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2007.05486.x · Cited by 77
Extinction in the developing rat: An examination of renewal effects
Yap, C. S. L., & Richardson, R. (2007) · Developmental Psychobiology · doi:10.1002/dev.20244 · Cited by 76
Attentional, associative, and configural mechanisms in extinction.
Larrauri, J. A., & Schmajuk, N. A. (2008) · Psychological Review · doi:10.1037/0033-295x.115.3.640 · Cited by 74
Context Learning in the Rodent Hippocampus
Fuhs, M. C., & Touretzky, D. S. (2007) · Neural Computation · doi:10.1162/neco.2007.19.12.3173 · Cited by 62
Capaldi, E. J. (1958) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0047677 · Cited by 61
Latent Attractors: A Model for Context-Dependent Place Representations in the Hippocampus
Doboli, S., Minai, A. A., & Best, P. J. (2000) · Neural Computation · doi:10.1162/089976600300015484 · Cited by 60
Excitation and inhibition in unblocking.
Holland, P. C. (1988) · PubMed · Cited by 59
Development of spatial firing in the hippocampus of young rats
Martin, P. D., & Berthoz, A. (2002) · Hippocampus · doi:10.1002/hipo.10021 · Cited by 55
Model Uncertainty in Classical Conditioning
Courville, A. C., Gordon, G. J., Touretzky, D. S., & Daw, N. D. (2003) · Cited by 51
Similarity and Discrimination in Classical Conditioning: A Latent Variable Account
Courville, A. C., Daw, N. D., & Touretzky, D. S. (2004) · Cited by 50
Nadel, L. (1995) · Hippocampus · doi:10.1002/hipo.450050309 · Cited by 35
Statistical Models of Conditioning
Dayan, P., & Long, T. W. (1997) · Cited by 34
Latent inhibition in the developing rat: An examination of context‐specific effects
Yap, C. S. L., & Richardson, R. (2005) · Developmental Psychobiology · doi:10.1002/dev.20074 · Cited by 34
The Effect of Different Amounts of Alternating Partial Reinforcement on Resistance to Extinction
Capaldi, E. J. (1957) · The American Journal of Psychology · doi:10.2307/1419584 · Cited by 28
Courville, A. C., & Daw, N. D. (2007) · neural information processing systems · Cited by 20
A latent cause theory of classical conditioning
Touretzky, D. S., & Courville, A. C. (2006) · Cited by 9
Cited by
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
“Using models that allow for the learning of such latent variables provides a better account of standard extinction phenomena 134 and predicts that stable unlearning can in fact occur as long as there is no obvious sudden switch, which was verified experimentally 135 .”
Orbitofrontal Cortex as a Cognitive Map of Task Space
Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.005 · Cited by 885
“Consequently, similar to the model of reversal learning, we modeled extinction with a twostate system (see also Gershman et al., 2010).”
InfluentialSelf-citationFrom reinforcement learning models to psychiatric and neurological disorders
Maia, T. V., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2723 · Cited by 722
“may be relevant for disorders that involve serotonergic abnormalities; and models of Pavlovian conditioning and extinctio”
BackgroundA tutorial on Bayesian nonparametric models
Gershman, S. J., & Blei, D. M. (2012) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2011.08.004 · Cited by 605
“A number of recent authors have extended this work (Griffiths et al., 2007; Heller et al., 2009; Sanborn et al., 2010) and applied it to other domains, such as classical conditioning (Gershman et al., 2010).”
Self-citationReasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making
Collins, A., & Koechlin, E. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001293 · Cited by 558
“new task sets for driving action. This issue raises a computational problem that statistical learning models based on Dirichlet process mixtures address”
Background
Metadata
- Authors
- Gershman, S. J., Blei, D. M., & Niv, Y.
- Affiliations
- Princeton University (US)
- Venue
- Psychological Review
- Published
- 2010
- Volume
- 117
- Issue
- 1
- Pages
- 197-209
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- OpenAlex
- W2086710210
- Cited by
- 386
- References
- 74
- Citations by year
- 2026: 26 · 2025: 29 · 2024: 38 · 2023: 27 · 2022: 24 · 2021: 33 · 2020: 22 · 2019: 32 · 2018: 21 · 2017: 26 · 2016: 29 · 2015: 17 · 2014: 15 · 2013: 16 · 2012: 15
- FWCI
- 8.79
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Review (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Google and Microsoft Research
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- context-dependent learning, extinction, reinforcement learning, state classification, Bayesian inference, latent inhibition, renewal, hippocampus, age differences
- MeSH terms
- Bayes Theorem, Conditioning, Classical, Extinction, Psychological, Hippocampus, Humans, Learning, Models, Psychological, Semantics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.