Surprised at All the Entropy: Hippocampal, Caudate and Midbrain Contributions to Learning from Prediction Errors
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Influential concepts in neuroscientific research cast the brain a predictive machine that revises its predictions when they are violated by sensory input. This relates to the predictive coding account of perception, but also to learning. Learning from prediction errors has been suggested for take place in the hippocampal memory system as well as in the basal ganglia. The present fMRI study used an action-observation paradigm to investigate the contributions of the hippocampus, caudate nucleus and midbrain dopaminergic system to different types of learning: learning in the absence of prediction errors, learning from prediction errors, and responding to the accumulation of prediction errors in unpredictable stimulus configurations. We conducted analyses of the regions of interests' BOLD response towards these different types of learning, implementing a bootstrapping procedure to correct for false positives. We found both, caudate nucleus and the hippocampus to be activated by perceptual prediction errors. The hippocampal responses seemed to relate to the associative mismatch between a stored representation and current sensory input. Moreover, its response was significantly influenced by the average information, or Shannon entropy of the stimulus material. In accordance with earlier results, the habenula was activated by perceptual prediction errors. Lastly, we found that the substantia nigra was activated by the novelty of sensory input. In sum, we established that the midbrain dopaminergic system, the hippocampus, and the caudate nucleus were to different degrees significantly involved in the three different types of learning: acquisition of new information, learning from prediction errors and responding to unpredictable stimulus developments. We relate learning from perceptual prediction errors to the concept of predictive coding and related information theoretic accounts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 3×
“cortical responses [11], especially in primary sensory cortices [9]”
BackgroundA theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“that is usually associated with reward related learning”
BackgroundLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
The hippocampal-VTA loop: controlling the entry of information into long-term memory
Lisman, J. E., & Grace, A. A. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.002 · Cited by 1,983
Learning and Memory Functions of the Basal Ganglia
Packard, M. G., & Knowlton, B. J. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142937 · Cited by 1,717 · cited 1×
“Neuroscientific research on learning has discussed the interplay and competition of two learning systems [23–27].”
BackgroundReward representations and reward-related learning in the human brain: Insights from neuroimaging
O’Doherty, J. P. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.10.016 · Cited by 1,426
Norman, K. A., & O'Reilly, R. C. (2003) · Psychological Review · doi:10.1037/0033-295x.110.4.611 · Cited by 1,312
Multiple reward signals in the brain
Schultz, W. (2000) · Nature Reviews Neuroscience · doi:10.1038/35044563 · Cited by 1,303
Object perception as Bayesian inference
Kersten, D., Mamassian, P., & Yuille, A. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.142005 · Cited by 1,269 · cited 3×
“bility of sensory activity at the level below [8,11,13,14], for the”
BackgroundItem, context and relational episodic encoding in humans
Davachi, L. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.10.012 · Cited by 1,138
Predictive coding: an account of the mirror neuron system
Kilner, J. M., Friston, K. J., & Frith, C. D. (2007) · Cognitive Processing · doi:10.1007/s10339-007-0170-2 · Cited by 1,134 · cited 2×
“Predictive coding is usually regarded to deal with current, not anticipated sensory input [12,5].”
BackgroundCompetition among multiple memory systems: converging evidence from animal and human brain studies
Poldrack, R. A., & Packard, M. G. (2003) · Neuropsychologia · doi:10.1016/s0028-3932(02)00157-4 · Cited by 880 · cited 1×
“Neuroscientific research on learning has discussed the interplay and competition of two learning systems [23–27].”
BackgroundA Hierarchy of Time-Scales and the Brain
Kiebel, S. J., Daunizeau, J., & Friston, K. J. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000209 · Cited by 694 · cited 1×
“hierarchy [8,9,11,16].”
Neural repetition suppression reflects fulfilled perceptual expectations
Summerfield, C., Trittschuh, E. H., Monti, J. M., Mesulam, M.-M., & Egner, T. (2008) · Nature Neuroscience · doi:10.1038/nn.2163 · Cited by 692 · cited 2×
“To resurrect the picture, the brain builds models of likely perceptions [11,14,50,51].”
BackgroundStatistically optimal perception and learning: from behavior to neural representations
Fiser, J., Berkes, P., Orbán, G., & Lengyel, M. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.003 · Cited by 672 · cited 1×
“proximity [7].”
Memory deficits for implicit contextual information in amnesic subjects with hippocampal damage
Chun, M. M., & Phelps, E. A. (1999) · Nature Neuroscience · doi:10.1038/12222 · Cited by 554
Predictive codes for forthcoming perception in the frontal cortex.
Summerfield, C., Egner, T., Greene, M., Koechlin, E., Mangels, J., & Hirsch, J. (2006) · Science · doi:10.1126/science.1132028 · Cited by 496
Huang, Y., & Rao, R. P. N. (2011) · WIREs Cognitive Science · doi:10.1002/wcs.142 · Cited by 470
Reinforcement learning or active inference?
Friston, K. J., Daunizeau, J., & Kiebel, S. J. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006421 · Cited by 443
O'Reilly, R. C., & Norman, K. A. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(02)02005-3 · Cited by 351
Striatal Prediction Error Modulates Cortical Coupling
den Ouden, H. E. M., Daunizeau, J., Roiser, J., Friston, K. J., & Stephan, K. E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4458-09.2010 · Cited by 304 · cited 2×
“The results of the present study substantially foster the alternative, not reward related, understanding of striatal prediction error signaling: the indication of prediction errors on a perceptual level, irrespective the presence of reward or punishment [42] and possibly associated with the amount of surprise that a p…”
BackgroundResult
Cited by
Precise minds in uncertain worlds: Predictive coding in autism.
Van de Cruys, S., Evers, K., Van der Hallen, R., Van Eylen, L., Boets, B., de-Wit, L., & Wagemans, J. (2014) · Psychological Review · doi:10.1037/a0037665 · Cited by 885
Pavlovian conditioning-induced hallucinations result from overweighting of perceptual priors
Powers, A. R., Mathys, C., & Corlett, P. R. (2017) · Science · doi:10.1126/science.aan3458 · Cited by 684
Affective Value in the Predictive Mind
Van De Cruys, S. (2017) · Theoretical Philosophy/MIND Group – JGU Mainz · doi:10.15502/9783958573253 · Cited by 100
Frontostriatal contribution to the interplay of flexibility and stability in serial prediction
Trempler, I., Schiffer, A.-M., El-Sourani, N., Ahlheim, C., Fink, G. R., & Schubotz, R. I. (2017) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01040 · Cited by 18
“With regard to the role of the striatum in immediate response selection, we hypothesized caudate activity in response to both switches and drifts (Schiffer et al., 2012).”
BackgroundSelf-citationTrempler, I., Bürkner, P.-C., El-Sourani, N., Binder, E., Reker, P., Fink, G. R., & Schubotz, R. I. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116674 · Cited by 6
Self-citation
Metadata
- Authors
- Schiffer, A.-M., Ahlheim, C., Wurm, M. F., & Schubotz, R. I.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-05-03
- Volume
- 7
- Issue
- 5
- Pages
- e36445
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W1999812934
- Cited by
- 65
- References
- 25
- FWCI
- 2.30
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.