CoChat
Sign Up
Cited by 164Neural and Behavioral Psychology StudiesVisual perception and processing mechanismsMemory and Neural Mechanisms

Mesolimbic confidence signals guide perceptual learning in the absence of external feedback

Open access
Guggenmos, Matthias, Wilbertz, Gregor, Hebart, Martin N, Sterzer, Philipp·2016·eLife
DOI 10.7554/elife.13388PMC PMC4821804
Open this paper in CoChat — free account

Abstract

It is well established that learning can occur without external feedback, yet normative reinforcement learning theories have difficulties explaining such instances of learning. Here, we propose that human observers are capable of generating their own feedback signals by monitoring internal decision variables. We investigated this hypothesis in a visual perceptual learning task using fMRI and confidence reports as a measure for this monitoring process. Employing a novel computational model in which learning is guided by confidence-based reinforcement signals, we found that mesolimbic brain areas encoded both anticipation and prediction error of confidence—in remarkable similarity to previous findings for external reward-based feedback. We demonstrate that the model accounts for choice and confidence reports and show that the mesolimbic confidence prediction error modulation derived through the model predicts individual learning success. These results provide a mechanistic neurobiological explanation for learning without external feedback by augmenting reinforcement models with confidence-based feedback.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7554/elife.13388

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The global broadcast of confidence can be used as a learning signal in lieu of external feedback, allowing online detection of errors and adjustments to behavior.

    “The global broadcast of confidence can also be used as a learning signal in lieu of external feedback-allowing agents the online detection of errors and consequent adjustments to behavior (Guggenmos et al. 2016).”

    From Metacognition and Confidence: A Review and Synthesis (2024) · cites this paper as Guggenmos et al. 2016

    The Introduction and Discussion describe confidence as an internal teaching signal in perceptual learning without external feedback; the Discussion says the model uses it for continuous adjustment. But the paper does not establish a global broadcast of confidence or online detection of actual errors: its prediction errors are mismatches between expected and current confidence. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 36

  1. Allefeld C Haynes JD 2014 Searchlight-based multi-voxel pattern analysis of fMRI by cross-validated MANOVA NeuroImage 89 345 357 10.1016/j.neuroimage.2013.11.043 24296330 · DOI
  2. Barto AG Sutton RS Brouwer PS 1981 Associative search network: A reinforcement learning associative memory Biological Cybernetics 40 201 211 10.1007/BF00453370 · DOI
  3. Berns GS McClure SM Pagnoni G Montague PR 2001 Predictability modulates human brain response to reward Journal of Neuroscience 21 2793 2798 11306631 PMC6762527
  4. Daniel R Pollmann S 2012 Striatal activations signal prediction errors on confidence in the absence of external feedback NeuroImage 59 3457 3467 10.1016/j.neuroimage.2011.11.058 22146752 · DOI
  5. Delgado MR Nystrom LE Fissell C Noll DC Fiez JA 2000 Tracking the hemodynamic responses to reward and punishment in the striatum Journal of Neurophysiology 84 3072 3077 11110834
  6. Duijvenvoorde AC Op de Macks ZA Overgaauw S Gunther Moor B Dahl RE Crone EA 2014 A cross-sectional and longitudinal analysis of reward-related brain activation: effects of age, pubertal stage, and reward sensitivity Brain and Cognition 89 3 14 10.1016/j.bandc.2013.10.005 24512819 · DOI · Paper page
  7. García-Pérez MA 1998 Forced-choice staircases with fixed step sizes: asymptotic and small-sample properties Vision Research 38 1861 1881 10.1016/S0042-6989(97)00340-4 9797963 · DOI
  8. Gibson EJ 1963 Perceptual learning Annual Review of Psychology 14 29 56 10.1146/annurev.ps.14.020163.000333 13947716 · DOI
  9. Gibson JJ Gibson EJ 1955 Perceptual learning; differentiation or enrichment? Psychological Review 62 32 41 10.1037/h0048826 14357525 · DOI · Paper page
  10. Gläscher J 2009 Visualization of group inference data in functional neuroimaging Neuroinformatics 7 73 82 10.1007/s12021-008-9042-x 19140033 · DOI
  11. Hebart MN Schriever Y Donner TH Haynes JD 2016 The Relationship between Perceptual Decision Variables and Confidence in the Human Brain Cerebral Cortex 26 10.1093/cercor/bhu181 25112281 · DOI
  12. Herzog MH Fahle M 1997 The role of feedback in learning a vernier discrimination task Vision Research 37 2133 2141 10.1016/S0042-6989(97)00043-6 9327060 · DOI
  13. Hulley S Cummings S Browner W Grady D Newman T 2013 Designing clinical research: an epidemiologic approach 4th eds Philadelphia, PA Lippincott Williams & Wilkins
  14. Hélie S Ell SW Ashby FG 2015 Learning robust cortico-cortical associations with the basal ganglia: An integrative review Cortex 64 123 135 10.1016/j.cortex.2014.10.011 25461713 · DOI
  15. Jaušovec N Jaušovec K 2010 Resting brain activity: differences between genders Neuropsychologia 48 3918 3925 10.1016/j.neuropsychologia.2010.09.020 20875436 · DOI
  16. Kaernbach C 1991 Simple adaptive testing with the weighted up-down method Perception & Psychophysics 49 227 229 10.3758/BF03214307 2011460 · DOI
  17. Kahnt T Grueschow M Speck O Haynes JD 2011 Perceptual learning and decision-making in human medial frontal cortex Neuron 70 549 559 10.1016/j.neuron.2011.02.054 21555079 · DOI
  18. Karni A Sagi D 1991 Where practice makes perfect in texture discrimination: evidence for primary visual cortex plasticity Proceedings of the National Academy of Sciences of the United States of America 88 4966 4970 10.1073/pnas.88.11.4966 2052578 PMC51788 · DOI
  19. Karni A Sagi D 1993 The time course of learning a visual skill Nature 365 250 252 10.1038/365250a0 8371779 · DOI
  20. Knutson B Adams CM Fong GW Hommer D 2001 Anticipation of increasing monetary reward selectively recruits nucleus accumbens Journal of Neuroscience 21 RC159 11459880 PMC6763187
  21. Kriegeskorte N Goebel R Bandettini P 2006 Information-based functional brain mapping Proceedings of the National Academy of Sciences of the United States of America 103 3863 3868 10.1073/pnas.0600244103 16537458 PMC1383651 · DOI · Paper page
  22. Kvam PD Pleskac TJ Yu S Busemeyer JR 2015 Interference effects of choice on confidence: Quantum characteristics of evidence accumulation Proceedings of the National Academy of Sciences 112 10645 10650 10.1073/pnas.1500688112 PMC4553814 26261322 · DOI
  23. Law CT Gold JI 2009 Reinforcement learning can account for associative and perceptual learning on a visual-decision task Nature Neuroscience 12 655 663 10.1038/nn.2304 19377473 PMC2674144 · DOI
  24. Mckee SP Westhe G 1978 Improvement in vernier acuity with practice Perception & Psychophysics 24 258 262 10.3758/BF03206097 704286 · DOI
  25. Petrov AA Dosher BA Lu ZL 2005 The dynamics of perceptual learning: an incremental reweighting model Psychological Review 112 715 743 10.1037/0033-295X.112.4.715 16262466 · DOI
  26. Petrov AA Dosher BA Lu ZL 2006 Perceptual learning without feedback in non-stationary contexts: data and model Vision Research 46 3177 3197 10.1016/j.visres.2006.03.022 16697434 · DOI
  27. Preuschoff K Bossaerts P Quartz SR 2006 Neural differentiation of expected reward and risk in human subcortical structures Neuron 51 381 390 10.1016/j.neuron.2006.06.024 16880132 · DOI · Paper page
  28. Ruigrok AN Salimi-Khorshidi G Lai MC Baron-Cohen S Lombardo MV Tait RJ Suckling J 2014 A meta-analysis of sex differences in human brain structure Neuroscience and Biobehavioral Reviews 39 34 50 10.1016/j.neubiorev.2013.12.004 24374381 PMC3969295 · DOI · Paper page
  29. Schlagenhauf F Rapp MA Huys QJ Beck A Wüstenberg T Deserno L Buchholz HG Kalbitzer J Buchert R Bauer M Kienast T Cumming P Plotkin M Kumakura Y Grace AA Dolan RJ Heinz A 2013 Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence Human Brain Mapping 34 1490 1499 10.1002/hbm.22000 22344813 PMC3731774 · DOI
  30. Schultz W Dayan P Montague PR 1997 A Neural Substrate of Prediction and Reward Science 275 1593 1599 10.1126/science.275.5306.1593 9054347 · DOI · Paper page
  31. Schultz W 2002 Getting Formal with Dopamine and Reward Neuron 36 241 263 10.1016/S0896-6273(02)00967-4 12383780 · DOI · Paper page
  32. Schwarze U Bingel U Badre D Sommer T 2013 Ventral striatal activity correlates with memory confidence for old- and new-responses in a difficult recognition test PloS One 8 e13388 10.1371/journal.pone.0054324 PMC3589408 23472064 · DOI
  33. Sieck WR 2003 Effects of choice and relative frequency elicitation on overconfidence: further tests of an exemplar-retrieval model Journal of Behavioral Decision Making 16 127 145 10.1002/bdm.438 · DOI
  34. Sniezek JA Paese PW Switzer FS 1990 The effect of choosing on confidence in choice Organizational Behavior and Human Decision Processes 46 264 282 10.1016/0749-5978(90)90032-5 · DOI
  35. Tafarodi RW Milne AB Smith AJ 1999 The Confidence of Choice: Evidence for an Augmentation Effect on Self-Perceived Performance Personality and Social Psychology Bulletin 25 1405 1416 10.1177/0146167299259006 · DOI
  36. Tzourio-Mazoyer N Landeau B Papathanassiou D Crivello F Etard O Delcroix N Mazoyer B Joliot M 2002 Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain NeuroImage 15 273 289 10.1006/nimg.2001.0978 11771995 · DOI · Paper page

Cited by

Sorted by citations

Authors
Guggenmos, M., Wilbertz, G., Hebart, M. N., & Sterzer, P.
Affiliations
  • Bernstein Center for Computational Neuroscience Berlin (DE)
  • Charité – Universitätsmedizin Berlin (DE)
  • University Medical Center Hamburg-Eppendorf (DE)
Corresponding author
Guggenmos, Matthias
ORCID
Venue
eLife
Publisher
eLife Sciences Publications, Ltd
Published
2016-03-29
Volume
5
ISSN
2050-084X (ISSN-L)
DOI
10.7554/elife.13388
PMC
PMC4821804
OpenAlex
W2323154394
Cited by
164
References
6
Citations by year
2026: 13 · 2025: 17 · 2024: 13 · 2023: 27 · 2022: 14 · 2021: 9 · 2020: 21 · 2019: 20 · 2018: 16 · 2017: 8 · 2016: 6
FWCI
10.61
Type
journal article
Language
en
Open access
Licence
Locations
  • eLife (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
  • Refubium (Universitätsbibliothek der Freien Universität Berlin) (submittedVersion, cc-by)
  • Universitätsbibliothek der FU Berlin Hochschulschriftenstelle u. Dokumentenserver (cc-by)
Funders
Deutsche Forschungsgemeinschaft (GRK 1589/1, STE 1430/6-1, STE 1430/7-1)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Keywords
perceptual learning, internal feedback, mesolimbic system, reinforcement learning, prediction error, anticipation, computational model, visual perception, individual differences in learning
MeSH terms
Adult, Computer Simulation, Feedback, Female, Humans, Learning, Limbic System, Magnetic Resonance Imaging, Models, Neurological, Reinforcement, Psychology, Young Adult, Anticipation, Psychological

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review