Distinct error-correcting and incidental learning of location relative to landmarks and boundaries
Open accessThe DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Associative reinforcement provides a powerful explanation of learned behavior. However, an unproven but long-held conjecture holds that spatial learning can occur incidentally rather than by reinforcement. Using a carefully controlled virtual-reality object-location memory task, we formally demonstrate that locations are concurrently learned relative to both local landmarks and local boundaries but that landmark-learning obeys associative reinforcement (showing “overshadowing” and “blocking” or “learned irrelevance”), whereas boundary-learning is incidental, showing neither overshadowing nor blocking nor learned irrelevance. Crucially, both types of learning occur at similar rates and do not reflect differences in levels of performance, cue salience, or instructions. These distinct types of learning likely reflect the distinct neural systems implicated in processing of landmarks and boundaries: the striatum and hippocampus, respectively [Doeller CF, King JA, Burgess N (2008) Proc Natl Acad Sci USA 105:5915–5920]. In turn, our results suggest the use of fundamentally different learning rules by these two systems, potentially explaining their differential roles in procedural and declarative memory more generally. Our results suggest a privileged role for surface geometry in determining spatial context and support the idea of a “geometric module,” albeit for location rather than orientation. Finally, the demonstration that reinforcement learning applies selectively to formally equivalent aspects of task-performance supports broader consideration of two-system models in analyses of learning and decision making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 9,661
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
(1978) · UA Campus Repository (The University of Arizona) · Cited by 8,010
Cognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641
Place navigation impaired in rats with hippocampal lesions
Morris, R. G. M., Garrud, P., Rawlins, J. N. P., & O'Keefe, J. (1982) · Nature · doi:10.1038/297681a0 · Cited by 6,340 · cited 1×
“Thus, rats in the water maze task use the boundary to locate the platform (39), a task that is hippocampal-dependent (40), and boundary-learning did not show blocking in our study.”
BackgroundClassical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 2×
“1, but note that these predictions also apply to other formulations of associative learning based on an error-correcting principle (2, 4, 26).”
BackgroundResultA theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Punishment and aversive behavior
Campbell, B. A., & Church, R. M. (1969) · Appleton-Century-Crofts eBooks · Cited by 1,989
A neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541
Packard, M. G., & McGaugh, J. L. (1996) · Neurobiology of Learning and Memory · doi:10.1006/nlme.1996.0007 · Cited by 1,492
Predictability, surprise, attention, and conditioning
Kamin, L. J. (1967) · NASA Technical Reports Server (NASA) · Cited by 1,483
Structure and function of declarative and nondeclarative memory systems
Squire, L. R., & Zola, S. M. (1996) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.93.24.13515 · Cited by 1,327 · cited 2×
“This produces incidental learning or ‘‘learning by observation’’ appropriate with maintenance of a flexible mental model (17), ‘‘mediating representation’’ (19), or ‘‘cognitive map’’ (6, 12), and with efficient encoding of experience into episodic memory (15, 16).”
BackgroundActions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217
A purely geometric module in the rat's spatial representation
Cheng, K. (1986) · Cognition · doi:10.1016/0010-0277(86)90041-7 · Cited by 1,169
Geometric determinants of the place fields of hippocampal neurons
O'Keefe, J., & Burgess, N. (1996) · Nature · doi:10.1038/381425a0 · Cited by 1,166
Self-citationInteractive memory systems in the human brain
Poldrack, R. A., Clark, J., Paré-Blagoev, E. J., Shohamy, D., Creso Moyano, J., Myers, C., & Gluck, M. A. (2001) · Nature · doi:10.1038/35107080 · Cited by 1,132
A model for stimulus generalization in Pavlovian conditioning.
Pearce, J. M. (1987) · Psychological Review · doi:10.1037//0033-295x.94.1.61 · Cited by 1,046
From Conditioning to Conscious Recollection
Eichenbaum, H., & Cohen, N. J. (2004) · doi:10.1093/acprof:oso/9780195178043.001.0001 · Cited by 1,026
Neurobiology of learning and memory
Delacour, J. (1985) · Neuropsychologia · doi:10.1016/0028-3932(85)90073-9 · Cited by 1,019
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018
The hippocampus and contextual retrieval of information from memory: A theory
Hirsh, R. (1974) · Behavioral Biology · doi:10.1016/s0091-6773(74)92231-7 · Cited by 957
Contemporary Animal Learning Theory
LoLordo, V. M., & Dickinson, A. (1982) · The American Journal of Psychology · doi:10.2307/1422669 · Cited by 888
Reward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855
Modularity and development: the case of spatial reorientation
Hermer, L., & Spelke, E. (1996) · Cognition · doi:10.1016/s0010-0277(96)00714-7 · Cited by 673
Planning for the future by western scrub-jays
Raby, C. R., Alexis, D. M., Dickinson, A., & Clayton, N. S. (2007) · Nature · doi:10.1038/nature05575 · Cited by 670
Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory
Doeller, C. F., King, J. A., & Burgess, N. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0801489105 · Cited by 521
Self-citationHead direction cells and the neurophysiological basis for a sense of direction
Taube, J. S. (1998) · Progress in Neurobiology · doi:10.1016/s0301-0082(98)00004-5 · Cited by 453
Morris, R. G. M., & Frey, U. (1997) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1997.0136 · Cited by 364
Le Pelley, M. E. (2004) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/02724990344000141 · Cited by 360 · cited 1×
“Other potential explanations are that poor predictors during initial learning will have an increased learning rate during compound learning (4), that the presence of a good predictor will aid the learning to a poor predictor by increasing reward frequency (‘‘feature enhancement’’) (34), that a cue’s associability vari…”
ResultHippocampal lesions disrupt navigation based on cognitive maps but not heading vectors
Pearce, J. M., Roberts, A. D. L., & Good, M. (1998) · Nature · doi:10.1038/23941 · Cited by 249
Failure of Centrally Placed Objects to Control the Firing Fields of Hippocampal Place Cells
Cressant, A., Muller, R. U., & Poucet, B. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-07-02531.1997 · Cited by 210
Learning about environmental geometry: An associative model.
Miller, N. Y., & Shettleworth, S. J. (2007) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.33.3.191 · Cited by 166
Brandon, S. E., Vogel, E. H., & Wagner, A. R. (2000) · Behavioural Brain Research · doi:10.1016/s0166-4328(99)00185-0 · Cited by 160
Geometric determinants of human spatial memory
Hartley, T., Trinkler, I., & Burgess, N. (2004) · Cognition · doi:10.1016/j.cognition.2003.12.001 · Cited by 158
Self-citationBlocking in the spatial domain.
Rodrigo, T., Chamizo, V. D., McLaren, I. P. L., & Mackintosh, N. J. (1997) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037//0097-7403.23.1.110 · Cited by 139
Pearce, J. M., Good, M. A., Jones, P. M., & McGregor, A. (2004) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/0097-7403.30.2.135 · Cited by 133
Potentiation, overshadowing, and blocking of spatial learning based on the shape of the environment.
Pearce, J. M., Graham, M., Good, M. A., Jones, P. M., & McGregor, A. (2006) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/0097-7403.32.3.201 · Cited by 130
Directional control of hippocampal place fields
Jeffery, K. J., Donnett, J. G., Burgess, N., & O'Keefe, J. M. (1997) · Experimental Brain Research · doi:10.1007/s002210050206 · Cited by 109
Self-citationExcitatory and inhibitory conditioning following uncorrelated presentations of CS and UCS
Baker, A. G., & Mackintosh, N. J. (1977) · Animal Learning & Behavior · doi:10.3758/bf03209246 · Cited by 106
Computational models of the hippocampal region: linking incremental learning and episodic memory
Gluck, M. A., Meeter, M., & Myers, C. E. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00105-0 · Cited by 103
Blocking in human place learning: Evidence from virtual navigation
Hamilton, D. A., & Sutherland, R. J. (1999) · Psychobiology · doi:10.3758/bf03332140 · Cited by 91
Hayward, A., McGregor, A., Good, M. A., & Pearce, J. M. (2003) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/02724990244000214 · Cited by 88
Chamizo, V. D. (2003) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/02724990244000205 · Cited by 82
Interaction between piloting and beacon homing by rats in a swimming pool.
Redhead, E. S., Roberts, A., Good, M. A., & Pearce, J. M. (1997) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.23.3.340 · Cited by 74
Blocking in the spatial domain with arrays of discrete landmarks.
Biegler, R., & Morris, R. G. M. (1999) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037//0097-7403.25.3.334 · Cited by 72
Place navigation in the morris water maze under minimum and redundant extra-maze cue conditions
Fenton, A. A., Arolfo, M. P., & Bureš, J. (1994) · Behavioral and Neural Biology · doi:10.1016/s0163-1047(05)80016-0 · Cited by 52
Effect of following an excitatory-inhibitory compound with an intermediate reinforcer.
Rescorla, R. A. (2002) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.28.2.163 · Cited by 22
Time cues block the CS, but the CS does not block time cues.
Williams, D. A., & LoLordo, V. M. (1995) · PubMed · Cited by 20
Cortico-hippocampal communication by way of parallel parahippocampal-subicular pathways.
Lavenex (2000) · Pure Amsterdam UMC · doi:10.1002/1098-1063(2000)10:4 · Cited by 1
Cited by
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
The hippocampus and memory: insights from spatial processing
Bird, C. M., & Burgess, N. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2335 · Cited by 1,313
Self-citationSpatial Cognition and the Brain
Burgess, N. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1440.002 · Cited by 614
Self-citationSpace in the brain: How the hippocampal formation supports spatial cognition
Hartley, T., Lever, C., Burgess, N., & O'Keefe, J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0510 · Cited by 495
“particularly necessary for boundary-based, rather than landmark-based, spatial lear ning in humans [112114]; and 2) boundary-based learning may follow different learning rules tha n landmark-based learning, although this is controversial [112, 113, 115117].”
BackgroundSelf-citationParallel striatal and hippocampal systems for landmarks and boundaries in spatial memory
Doeller, C. F., King, J. A., & Burgess, N. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0801489105 · Cited by 521
Self-citation
Metadata
- Authors
- Doeller, C. F., & Burgess, N.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2008-04-15
- Volume
- 105
- Issue
- 15
- Pages
- 5909-5914
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2145405131
- Cited by
- 283
- References
- 55
- FWCI
- 10.23
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.