Behavioral and Neural Properties of Social Reinforcement Learning

Pre-task likeability ratings were equivalent for all three peers.

“Whereas pre-task likeability ratings were equivalent for all three peers (p’s > 0.48)”

After the task, the Frequent and Continuous peers were rated as more likeable than the rarely reinforcing peer.

“after the task the Frequent (t(39) = −2.26, p < 0.03) and Continuous (t(39) = −2.68, p < 0.02) peers were rated as more likeable than the rarely reinforcing peer”

Participants responded more quickly to faces of peers who provided more frequent positive social reinforcement.

“participants responded more quickly to faces of peers who provided more frequent positive social reinforcement”

Rostral anterior cingulate cortex activity tracked positively with modulations of expected value of the cues (peers).

“Rostral anterior cingulate cortex activity tracked positively with modulations of expected value of the cues (peers).”

There was not a significant difference in likeability ratings after the task between the Frequent and Continuous peers.

“though there was not a significant difference in likeability ratings after the task between the Frequent and Continuous peers (t(39) = −0.69, p > 0.49).”

Post-task likeability ratings decreased linearly with decreasing interaction probability, such that peers who interacted less with the participant were rated as less likeable.

“Post-hoc analyses indicated that post-task ratings decreased linearly with decreasing interaction probability, such that peers who interacted less with the participant were rated as less likeable (linear term F(1,39) = 7.17, p < 0.02).”

Enhanced social preference for peers was modulated by the probability of reinforcement experienced during the experiment, as indicated by a significant interaction between reinforcement probability and time on likeability ratings.

“Enhanced social preference for peers was modulated by the probability of reinforcement experienced during the experiment, as indicated by a significant interaction between reinforcement probability and time (pre-interaction, post-interaction) on likeability ratings (F(2,78) = 5.48, p < 0.01; Figure 2A).”

Response accuracy was not significantly modulated by the task conditions.

“Response accuracy was not significantly modulated by the task conditions (main effects of reinforcement probability, time and interactions p’s > 0.29).”

Ventral striatum and orbital frontal cortex activity correlated positively with forming expectations about receiving social reinforcement.

“Modeling trial-by-trial learning showed ventral striatum and orbital frontal cortex activity correlated positively with forming expectations about receiving social reinforcement.”

Attractiveness ratings were not significantly modulated by task conditions.

“Attractiveness ratings were not significantly modulated by task conditions (main effects of reinforcement probability, time and interactions p’s > 0.09).”

Participants rated peers who provided more frequent positive social reinforcement as more likeable.

“rated them as more likeable”

• Also stated in: § Behavioral · plain

  “after the task the Frequent (t(39) = −2.26, p < 0.03) and Continuous (t(39) = −2.68, p < 0.02) peers were rated as more likeable than the rarely reinforcing peer”

There was no main effect of probability of social reinforcement when collapsing across time.

“there was no main effect of probability of social reinforcement when collapsing across time (F(2,84) = 1.43, p > 0.25).”

Overall, participants were faster during the late versus early trials.

“Overall, participants were faster during the late versus early trials (F(1,42) = 15.21, p < 0.01)”

The striatum is sensitive to expectations about receiving social feedback.

“Our finding that the striatum is sensitive to expectations about receiving social feedback converges with other work targeting the neural mechanisms of social learning.”

The majority (93%) of participants were unable to articulate the reinforcement patterns.

“The majority (93%) of participants were unable to articulate the reinforcement patterns, suggesting little if any explicit awareness of the reinforcement contingencies.”

Participants responded faster to peers who provided positive social reinforcement more often.

“In the current study, we observed faster responses to peers who provided positive social reinforcement more often, similar to studies where participants respond more quickly to cues that reliably predict receiving a primary or secondary reward (O’Doherty et al., 2006; Spicer et al., 2007).”

• cites O’Doherty et al., 2006Not checked yet

  Not checked in the free run.

• cites Spicer et al., 2007Not checked yet

  Not checked in the free run.

There was no difference in the Frequent condition (unchanged during reversal) reaction times between the last run and the reversal run.

“There was also no difference in the Frequent condition (unchanged during reversal) reaction times between the last run and the reversal run (t(42) = −1.48, p > 0.15).”

There was no difference in reaction times in the early trials between reinforcement conditions.

“Post-hoc t-tests showed that whereas there was no difference in reaction times in the early trials (all p’s > 0.41)”

Response latencies to the cue varied as participants learned the reinforcement contingency outcomes associated with each peer, as indicated by a significant interaction between probability of reinforcement and time.

“Response latencies to the cue varied as participants learned the reinforcement contingency outcomes associated with each peer, as indicated by a significant interaction between probability of reinforcement and time (F(2,84) = 3.98 p < 0.03; Figure 2B).”

Group analysis of the cue phase of trials that tracked positively with modulations of expected value (Vt) identified greater activity in the rostral anterior cingulate cortex with larger expected value.

“Specifically, group analysis of the cue phase of trials that tracked positively with modulations of expected value (Vt) identified greater activity in the rostral anterior cingulate cortex with larger expected value (see Figure 4).”

Prediction error signals (δt) were positively associated with activity in the rostral anterior cingulate cortex, ventral striatum, anterior insula, and OFC.

“As indexed by the prediction error parametric regressor, prediction error signals (δt) were positively associated with activity in the rostral anterior cingulate cortex, ventral striatum, anterior insula, and OFC (see Table 1 & Figure 3).”

No regions within the frontostriatal circuitry of interest demonstrated negative correlations with expected value at corrected thresholding.

“No regions within the frontostriatal circuitry of interest demonstrated negative correlations with expected value (i.e. brain regions sensitive to lower expected values) at corrected thresholding.”

Individuals were faster during the late trials of the Frequent reinforcement condition compared to the Rare reinforcement condition.

“individuals were faster during the late trials of the Frequent reinforcement condition (t(42) = 2.49, p < 0.02), as compared to the Rare reinforcement condition.”

No other regions survived whole brain correction for expected value.

“No other regions survived whole brain correction.”

Participants were faster on the subsequent trial after not receiving positive reinforcement versus when they had received positive reinforcement.

“We found that participants were faster on the subsequent trial after not receiving positive reinforcement (mean z-score RT: −0.01, SD: 0.13) versus when they had received positive reinforcement (mean z-score RT: 0.07, SD: 0.1; t(42) = 2.86, p < 0.01).”

• Also stated in: § Discussion · plain

  “In addition, participants showed faster reaction times after trials that did not provide positive social reinforcement.”

There was a significant reaction time speeding when the Rare condition switched to delivering Continuous reinforcement.

“Post-hoc tests showed a significant reaction time speeding when the Rare condition switched to delivering Continuous reinforcement (t(42) = 3.13, p < 0.01).”

There was an interaction between time (6th run versus reversal) and reinforcement probability (rarely reinforcing versus continuously reinforcing) on reaction times.

“the interaction between time (6th run versus reversal) and reinforcement probability (rarely reinforcing versus continuously reinforcing) on reaction times (F(1,42) = 10.15, p < 0.01; Figure 2C).”

Violations in expected social interaction are tightly coupled with changes in ventral striatal activity.

“Using a simple Rescorla-Wagner learning model, we show that violations in expected social interaction are tightly coupled with changes in ventral striatal activity.”

There was a trend for responses being faster in the Continuous reinforcement condition than in the Rare reinforcement condition.

“There was a trend for responses being faster in the Continuous reinforcement condition (t(42) = 2.01, p < 0.06) than in the Rare reinforcement condition.”

Acceptance by one's peers has a fundamental influence in altering subsequent behavior.

“This work highlights the fundamental influence of acceptance by one’s peers in altering subsequent behavior.”

As cues become reliably associated with receipt of a reward, manual responses to these cues quicken over time.

“Studies have shown that as cues become reliably associated with receipt of a reward, manual responses to these cues quicken over time (O’Doherty et al., 2006; Spicer et al., 2007)”

• cites O’Doherty et al., 2006Not checked yet

  Not checked in the free run.

• cites Spicer et al., 2007Not checked yet

  Not checked in the free run.

The orbital frontal cortex has been implicated in prior studies of prediction error learning.

“As the OFC has been implicated in prior studies of prediction error learning (Berns et al., 2001; Takahashi et al., 2009), we selected this region as an a priori structure in which to search for learning signals.”

• cites Berns et al., 2001 sourceNot checked yet

  Not checked in the free run.

• cites Takahashi et al., 2009Not checked yet

  Not checked in the free run.

The findings across three levels of analysis-social preferences, response latencies and modeling neural responses-are consistent with reinforcement learning theory and non-human primate electrophysiological studies of reward.

“Together, the findings across three levels of analysis - social preferences, response latencies and modeling neural responses – are consistent with reinforcement learning theory and non-human primate electrophysiological studies of reward.”

Changes in likeability ratings and response latencies did not appear to be conscious behavioral choices.

“The changes in likeability ratings and response latencies did not appear to be conscious behavioral choices.”

Learning social cue values drives changes in behavior that may differ or conflict with the cognitive demands of the situation (e.g. task demands).

“our findings suggest that learning social cue values drives changes in behavior that may differ or conflict with the cognitive demands of the situation (e.g. task demands).”

The neural systems engaged while forming social expectations are similar to those involved in basic reward learning.

“Within a reinforcement learning framework, we developed a novel social paradigm and demonstrated that the neural systems engaged while forming social expectations are similar to those involved in basic reward learning.”

Participants learned to differentiate each of the cues (peers) by their distinct reinforcement outcomes.

“Over the course of the experiment, participants learned to differentiate each of the cues (peers) by their distinct reinforcement outcomes.”

The anterior cingulate cortex may play a role in processing behavioral tendencies towards learned social cues.

“the findings in the anterior cingulate cortex may suggest a role for this region in processing behavioral tendencies towards learned social cues.”

Similar mechanisms underlie basic reinforcement learning and our ability to rapidly and flexibly update our expectations during interactions with others.

“Overall, the findings suggest that similar mechanisms underlie basic reinforcement learning and our ability to rapidly and flexibly update our expectations during interactions with others, which enables us to effectively navigate the social environment.”

An orbital frontostriatal circuit showed significantly greater activity associated with the unexpected outcomes of either receiving or not receiving positive social reinforcement.

“Together these findings delineate an orbital frontostriatal circuit showing significantly greater activity associated with the unexpected outcomes of either receiving or not receiving positive social reinforcement.”

Changes in responses in the current study are thought to index differences in approach behaviors that are based upon learning from a prior history of social feedback.

“while in the current study changes in responses are thought to index differences in approach behaviors that are based upon learning from a prior history of social feedback.”

Brief, positive, social interactions can significantly shape social learning across three discrete measures - social preferences, behavioral actions, and neural activity.

“Our findings provide direct evidence for how brief, positive, social interactions can significantly shape social learning across three discrete measures - social preferences, behavioral actions, and neural activity.”

After short interactions with others, social preferences and actions can be altered.

“After short interactions with others, social preferences and actions can be altered, highlighting the significance of social acceptance in biasing behavior.”

The orbital frontostriatal circuit plays a role in generating learning signals from positive social reinforcement.

“Together, these findings support a role for the orbital frontostriatal circuit in generating learning signals from positive social reinforcement and provide a neural basis for how feedback during a social interaction is flexibly updated in order to inform subsequent social expectations.”

Feedback during a social interaction is flexibly updated in order to inform subsequent social expectations.

“provide a neural basis for how feedback during a social interaction is flexibly updated in order to inform subsequent social expectations.”

Prior positive interactions with others shape our expectations for future interactions.

“Our results complement these studies by demonstrating a neural mechanism for how prior positive interactions with others shape our expectations for future interactions.”

BOLD fluctuations in these regions tracked learning signals reflecting reinforcement expectancies.

“The parametric values in the general linear model encompassed positive and negative prediction errors, demonstrating that the BOLD fluctuations in these regions tracked learning signals reflecting reinforcement expectancies.”

Participants had implicitly learned the reinforcement contingencies.

“Evidence that participants had implicitly learned the contingencies was further supported by the interaction between time (6th run versus reversal) and reinforcement probability (rarely reinforcing versus continuously reinforcing) on reaction times (F(1,42) = 10.15, p < 0.01; Figure 2C).”

There was little if any explicit awareness of the reinforcement contingencies.

“suggesting little if any explicit awareness of the reinforcement contingencies.”

Formal computational models of reinforcement learning apply to secondary reinforcement learning in the social domain.

“Moreover, we show that formal computational models of reinforcement learning apply to secondary reinforcement learning in the social domain.”

The neural circuitry involved in forming prediction error signals about receiving social reinforcement, including the ventral striatum and orbital frontal cortex, overlaps with circuitry that subserves learning about other types of rewards (e.g. food or money).

“We demonstrate that the neural circuitry involved in forming prediction error signals about receiving social reinforcement, including the ventral striatum and orbital frontal cortex, overlaps with circuitry that subserves learning about other types of rewards (e.g. food or money).”