Vicarious Reinforcement Learning Signals When Instructing Others
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Reinforcement learning (RL) theory posits that learning is driven by discrepancies between the predicted and actual outcomes of actions (prediction errors [PEs]). In social environments, learning is often guided by similar RL mechanisms. For example, teachers monitor the actions of students and provide feedback to them. This feedback evokes PEs in students that guide their learning. We report the first study that investigates the neural mechanisms that underpin RL signals in the brain of a teacher. Neurons in the anterior cingulate cortex (ACC) signal PEs when learning from the outcomes of one's own actions but also signal information when outcomes are received by others. Does a teacher's ACC signal PEs when monitoring a student's learning? Using fMRI, we studied brain activity in human subjects (teachers) as they taught a confederate (student) action–outcome associations by providing positive or negative feedback. We examined activity time-locked to the students' responses, when teachers infer student predictions and know actual outcomes. We fitted a RL-based computational model to the behavior of the student to characterize their learning, and examined whether a teacher's ACC signals when a student's predictions are wrong. In line with our hypothesis, activity in the teacher's ACC covaried with the PE values in the model. Additionally, activity in the teacher's insula and ventromedial prefrontal cortex covaried with the predicted value according to the student. Our findings highlight that the ACC signals PEs vicariously for others' erroneous predictions, when monitoring and instructing their learning. These results suggest that RL mechanisms, processed vicariously, may underpin and facilitate teaching behaviors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Empathy for Pain Involves the Affective but not Sensory Components of Pain
Singer, T., Seymour, B., O'Doherty, J., Kaube, H., Dolan, R. J., & Frith, C. D. (2004) · Science · doi:10.1126/science.1093535 · Cited by 3,929
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
The Neural Basis of Mentalizing
Frith, C. D., & Frith, U. (2006) · Neuron · doi:10.1016/j.neuron.2006.05.001 · Cited by 1,855
Behavioral Theories and the Neurophysiology of Reward
Schultz, W. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.56.091103.070229 · Cited by 1,637 · cited 1×
“The behavior of the student was modeled using a simple Rescorla-Wagner (R-W)-based reinforcement learning algorithm (Rescorla and Wagner, 1972), which has been extensively used to examine the behavioral and neural basis of arbitrary visuomotor associations (Dayan and Balleine, 2002; Schultz, 2006; Brovelli et al., 200…”
MethodToward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · cited 1×
“This method is a standard approach used in reinforcement learning theory (Sutton and Barto, 1981).”
MethodNeuronal Coding of Prediction Errors
Schultz, W., & Dickinson, A. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.473 · Cited by 1,427
Insula of the old world monkey. III: Efferent cortical output and comments on function
Mesulam, M. ‐., & Mufson, E. J. (1982) · Journal of Comparative Neurology · doi:10.1002/cne.902120104 · Cited by 1,013 · cited 1×
“In addition, the ACCg has mono- synaptic connections to the portions of the insula and the VmPFC that were found to covary with the student’s prediction in this study (Mesulam and Mufson, 1982; Mufson and Mesulam, 1982; Morecraft et al., 1992; Cavada et al., 2000).”
ResultAssociative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925
Cingulate cortex of the rhesus monkey: II. Cortical afferents
Vogt, B. A., & Pandya, D. N. (1987) · Journal of Comparative Neurology · doi:10.1002/cne.902620208 · Cited by 854 · cited 1×
“…midcingulate cortex) has strong connections to the posterior portions of the superior temporal sulcus, the temporal poles (Markowitsch et al., 1985; Seltzer and Pandya, 1989; Barbas et al., 1999), and the paracingulate cortex (Pandya et al., 1981; Vogt and Pandya, 1987; Petrides and Pandya, 2007).”
BackgroundOptimized EPI for fMRI studies of the orbitofrontal cortex
Deichmann, R., Gottfried, J. A., Hutton, C., & Turner, R. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00073-9 · Cited by 821
Cited by
The Anterior Cingulate Gyrus and Social Cognition: Tracking the Motivation of Others
Apps, M. A. J., Rushworth, M. F. S., & Chang, S. W. C. (2016) · Neuron · doi:10.1016/j.neuron.2016.04.018 · Cited by 559
“Furthermore, it makes specific, testable predictions about a key mechanism that may underpin variability in socio-cognitive abilities in health and disease.”
Self-citationResolving uncertainty in a social world
FeldmanHall, O., & Shenhav, A. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0590-x · Cited by 274
Disrupted prediction errors index social deficits in autism spectrum disorder
Balsters, J. H., Apps, M. A. J., Bolis, D., Lehner, R., Gallagher, L., & Wenderoth, N. (2017) · Brain · doi:10.1093/brain/aww287 · Cited by 85
“…receive a reward (Lockwood et al., 2015), the net-value (costbenefit) of another individual’s decision (Apps and Ramnani, 2014), and the discrepancy between another person’s expectations and the actual outcomes, i.e. social prediction errors (Behrens et al., 2008; Apps et al., 2012, 2013a, 2015).”
BackgroundSelf-citationBalsters, J. H., Mantini, D., Apps, M. A. J., Eickhoff, S. B., & Wenderoth, N. (2016) · NeuroImage: Clinical · doi:10.1016/j.nicl.2016.03.016 · Cited by 50
Self-citation
Metadata
- Authors
- Apps, M. A. J., Lesage, E., & Ramnani, N.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2015-02-18
- Volume
- 35
- Issue
- 7
- Pages
- 2904-2913
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2110359145
- Cited by
- 72
- References
- 26
- FWCI
- 5.16
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Vicarious Reinforcement Learning Signals When Instructing Others. https://app.cochat.ai/papers/10.1523/jneurosci.3669-14.2015
- The paper
- Apps, M. A. J., Lesage, E., & Ramnani, N. (2015). Vicarious Reinforcement Learning Signals When Instructing Others. The Journal of Neuroscience. https://doi.org/10.1523/jneurosci.3669-14.2015
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.