Representation of aversive prediction errors in the human periaqueductal gray

May combine two claims.

The temporal-difference learning model fitted participants' choices with learning rate α=0.63±0.26, softmax inverse temperature β=4.74±2.74, and negative log likelihood 65±21, comparable to similar reinforcement-learning studies.

“The analysis revealed learning rates (α=0.63±0.26), softmax inverse temperatures (β=4.74±2.74), and model fits (negative log likelihood = 65 ± 21) comparable to those found in similar studies of reinforcement learning20.”

• cites [20] Schönberg, T., Daw, N.D., Joel, D. & O'Doherty, J.P. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J. Neurosci. 27, 12860–12867 (2007). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The effect of pain on switching decayed exponentially over time, and reinforcement history from pain one to two trials back significantly affected switches (p < 0.05).

“The effects of pain on switching also decayed exponentially with time, as evidenced by the results of logistic regressions assessing the effects of reinforcement history (pain delivered 1 to 6 trials back) on switches (p < 0.05 for 1 to 2 trials back; see Fig. 1c).”

May combine two claims.

Activity correlating with model-based aversive prediction errors (greater activity for worse-than-expected outcomes) was found in left anterior insula, anterior and midcingulate cortices, right pre- and post-central gyri, right dorsolateral prefrontal cortex, and a midbrain cluster encompassing the periaqueductal gray.

“Activity correlating with model-based aversive PEs (greater activity for worse-than-expected outcomes) was found in several areas, as shown in Figure 1d. These included the left anterior insula, anterior and midcingulate cortices (ACC and MCC), the right pre- and post- central gyrii, the right dorsolateral prefrontal cortex (dlPFC), and a large cluster in the midbrain encompassing the periaqueduct”

May combine two claims.

Several regions correlated with prediction errors only on pain trials: left orbitofrontal cortex, anterior mid-cingulate cortex, dorsomedial prefrontal cortex, and a larger dorsal midbrain cluster comprising periaqueductal gray, tectum, dorsal raphe nucleus, nucleus cuneiformis, and red nucleus.

“Several regions identified in the conjunction analysis correlated with PEs only on pain trials (blue in Fig. 3a), including the left OFC, anterior MCC (aMCC), dorsomedial prefrontal cortex (dmPFC) and a larger dorsal midbrain cluster comprising the PAG, tectum, DRN, NCF and red nucleus (RN).”

May combine two claims.

Within that periaqueductal gray/tectum cluster, activity did not differ between highly predicted Pain and No-Stimulus outcomes (t(22)=0.56, p>0.4, odds in favor of the null 5.13), confirming Axiom 3.

“There was no activity difference between Pain and No-Stimulus trials when outcomes were highly predicted (t(22): 0.56, p > 0.4, odds in favor of the null: 5.13), thereby confirming Axiom #3.”

May combine two claims.

Left putamen, ventromedial prefrontal cortex, and right hippocampus showed increased activity when pain was expected to be avoided (low pain expectancy) but no differential response to Pain versus No-Stimulus outcomes.

“We identified clusters in the left putamen, vmPFC, and right hippocampus in which activity fit this profile, consistent with expectancy effects. These regions displayed increased activity when pain was expected to be avoided (low pain expectancy), but didn't respond differentially to pain vs. No-stimulus outcomes (green clusters in Fig. 3a).”

May combine two claims.

The three poorly fitting participants did so because one used a win-stay loose-shift strategy, one switched frequently after no pain due to an irrelevant strategy, and one had many missing responses.

“This was caused by complete reliance on a win-stay loose-shift strategy (1 subject), frequent switches in choice following absence of pain, which was caused by use of an irrelevant strategy (1 subject), or numerous missing responses (20% missing; 1 subject).”

May combine two claims.

In the most likely DCM, the vmPFC projects most directly to the PAG, and avoidance value is most closely linked to the putamen, which sends value information to the vmPFC.

“In this model, vmPFC projects most directly to PAG, and avoidance value is most closely related to the putamen, which transmits value information to vmPFC.”

• AssumptionThe authors state this as an assumption: Primary nociceptive afferents project directly to the PAG.
• AssumptionThe authors state this as an assumption: Expected avoidance value reaches the PAG through at least one of the three expected-value structures (vmPFC, putamen, hippocampus).
• AssumptionThe authors state this as an assumption: Prediction error signals can be sent directly to the OFC and aMCC, and indirectly to the dmPFC via either aMCC or OFC.

May combine two claims.

The best overall DCM featured direct putamen and midbrain inputs to OFC and aMCC, dmPFC effects routed through OFC, and noxious-input modulation of putamen→OFC, putamen→aMCC, and midbrain→aMCC connections.

“The best model overall is shown in Figure 4, and included a) direct connections from both putamen and midbrain to OFC and aMCC, with dmPFC effects mediated by OFC, and b) modulatory effects of noxious input to putamen → OFC, putamen → aMCC, and midbrain → aMCC connections.”

• AssumptionThe authors state this as an assumption: Primary nociceptive afferents project directly to the PAG.
• AssumptionThe authors state this as an assumption: Expected avoidance value reaches the PAG through at least one of the three expected-value structures (vmPFC, putamen, hippocampus).
• AssumptionThe authors state this as an assumption: Prediction error signals can be sent directly to the OFC and aMCC, and indirectly to the dmPFC via either aMCC or OFC.

May combine two claims.

The periaqueductal gray is a primary site for aversive prediction errors, contrary to previous theories that a single system drives appetitive and aversive prediction errors.

“Our data and connectivity models identify the PAG as a primary site for aversive PEs, in contrast to previous neuroimaging findings and theories that a single system drives appetitive and aversive PEs29.”

• cites [29] Brooks, A.M. & Berns, G.S. Aversive stimuli and loss in the mesocorticolimbic dopamine system. Trends Cogn. Sci. 17, 281–286 (2013). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study was approved by the Columbia University Institutional Review Board and all participants provided informed consent.

“All participants provided informed consent. The study was approved by the Columbia University Institutional Review Board.”

May combine two claims.

Study 3 results replicate and extend the findings of pain-related aversive prediction errors in the periaqueductal gray, showing sensitivity to painful stimulus intensity and to verbal instructions as well as predictive cues.

“Overall, results from this Supplementary experiment replicate and extend the findings of pain-related aversive prediction errors in the PAG, demonstrating sensitivity to the level of painful stimulus intensity and sensitivity to verbal instructions as well as predictive cues.”

May combine two claims.

The results indicate convergence between appetitive and aversive systems in value representations in ventromedial prefrontal cortex, and segregation between the two systems when expected values are integrated with ascending aversive unconditioned stimulus inputs in periaqueductal gray to generate prediction errors.

“Hence, our results seem to indicate both a convergence between appetitive and aversive systems in value representations in the vmPFC, and a segregation between the two systems when these expected values signals are integrated with ascending aversive unconditioned stimuli (US) inputs in the PAG to generate PE signals.”

Classified as probably not a substantial claim.

Statistical maps were corrected for multiple comparisons using cluster-level FWE at p<0.05 (two-tailed), with cluster-defining thresholds of p<0.001, p<0.01, and p<0.05 via AFNI's alphasim.

“Results were cluster-corrected (p<0.05, FWE, two-tailed) with cluster-defining thresholds of p <0.001, p<0.01 and p<0.05 using AFNI's alphasim.”

May combine two claims.

In the final dynamic causal model, prediction error and expected value signals from midbrain and putamen are transmitted to orbitofrontal cortex and anterior mid-cingulate cortex, with effects on dorsomedial prefrontal cortex mediated by orbitofrontal cortex.

“Then, PE and expected value signals from the midbrain and putamen are transmitted to OFC and aMCC, with effects on dmPFC mediated by OFC.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The a priori PAG region of interest was consistent with a recent meta-analysis of pain processing in the PAG.

“The PAG a priori ROI was constructed by aligning three overlapping 6-mm spheres along the central aqueduct ([0 –24 –4; 0 –26 –6; 0 –29 –8]), and closely matched the findings of a recent meta-analysis on pain-processing in the PAG (Linnman, Moulton, Barmettler, Becerra, & Borsook, 2012a).”

• cites Linnman, Moulton, Barmettler, Becerra, & Borsook, 2012aNot checked yet

  Not checked in the free run.

May combine two claims.

Findings in the larger midbrain cluster are consistent with aversive prediction error signals, although only the dorsal periaqueductal gray portion survived whole-brain correction in all three contrasts.

“Thus, findings in this larger midbrain cluster are consistent with aversive PE signals, though the dorsal PAG region was the only portion to survive whole-brain correction in all three contrasts.”

May combine two claims.

Periaqueductal gray neurons respond more strongly to unexpected than expected electric shocks, consistent with the fMRI findings.

“PAG neurons respond more strongly to unexpected vs. expected electric shocks18, consistent with our fMRI findings here.”

• cites [18] Johansen, J.P., Tarpley, J.W., LeDoux, J.E. & Blair, H.T. Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray. Nat. Neurosci. 13, 979–986 (2010). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The thermode was moved to a new skin spot after each run to minimize peripheral sensitization and habituation.

“In order to minimize the effects of peripheral sensitization/habituation, the thermode was moved to a new skin spot after each run.”

May combine two claims.

The anterior mid-cingulate cortex is critical for pain avoidance and is heavily connected to motor and premotor centers.

“The aMCC in particular is critical for pain avoidance18 and is heavily connected to motor and premotor centers (aMCC).”

• cites [18] Johansen, J.P., Tarpley, J.W., LeDoux, J.E. & Blair, H.T. Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray. Nat. Neurosci. 13, 979–986 (2010). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Ventromedial prefrontal cortex may act as a hub or convergence point for different types of expected value signals, such as experienced-based expected value computed in putamen and model-based value computed in caudate.

“This is consistent with recent findings that the vmPFC may act as a “hub” or “convergence point” for different types of expected value signals, such as experienced-based expected value signals computed in the putamen and model-based value signals computed in the caudate48.”

• cites [48] Wunderlich, K., Dayan, P. & Dolan, R.J. Mapping value based planning and extensively trained choice in the human brain. Nat. Neurosci. 15, 786–791 (2012). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The results are consistent with ventromedial prefrontal cortex as the most direct representation of value related to choice and learning and as most closely connected to periaqueductal gray in the retained dynamic causal model.

“Our results are consistent with the vmPFC as the most direct representation of value as related to choice and learning (Fig. 3 and Supplementary Fig. 4), and most closely connected to the PAG in the retained DCM model.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Three participants were excluded from analysis because of poor task performance.

“Three participants were excluded from the analysis because of their poor performance on task (see section on reinforcement model-based analysis below).”

May combine two claims.

In the final dynamic causal model, noxious input had direct effects on periaqueductal gray and modulatory effects on ascending putamen-to-orbitofrontal, putamen-to-anterior-mid-cingulate, and midbrain-to-anterior-mid-cingulate connections.

“Noxious input had direct effects on PAG, in keeping with the known anatomy of the ascending spinomesencephalic nociceptive pathway28, and modulatory effects on ascending putamen-to-OFC, putamen-to-aMCC, and midbrain-to-aMCC connections.”

• cites [28] Millan, M.J. The induction of pain: an integrative review. Prog. Neurobiol. 57, 1–164 (1999). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Thermal stimulation was calibrated individually to a level 7 on a 0-8 pain scale, ensuring the most painful temperature within tolerance.

“Temperatures were individually calibrated to be at a level 7 on a continuous scale ranging from 0 to 8 (0, no sensation; 1, nonpainful warmth; 2, low pain; 5, moderate pain; 8, maximum tolerable pain), during a practice session performed on a separate day prior to the imaging session. This procedure ensured that participants received the most painful level of the temperature within tolerance”

May combine two claims.

The periaqueductal gray receives monosynaptic inputs from nociceptive spinal projection neurons and top-down projections from ventromedial prefrontal cortex, positioning it as a potential comparator of bottom-up aversive sensations with expectations.

“The PAG receives monosynaptic inputs from both nociceptive spinal projection neurons28 and top-down projections from the vmPFC30, positioning it as a potential comparator of ‘bottom up’ aversive sensations with expectations.”

• cites [28] Millan, M.J. The induction of pain: an integrative review. Prog. Neurobiol. 57, 1–164 (1999). sourceNot checked yet

  Not checked in the free run.

• cites [30] Price, J.L. Definition of the orbital cortex in relation to specific connections with limbic and visceral structures and other cortical regions. Ann. NY Acad. Sci. 1121, 54–71 (2007). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The other three regions showed little evidence for expectancy effects on No-Stimulus trials (Bayes factors in favor of the null all > 2.24) and are more likely to reflect avoidance value updating or other motivational processes.

“The other three regions showed little evidence for expectancy effects on No-Stimulus trials (Bayes factors in favour of the null all > 2.24), and thus are more likely to reflect avoidance value updating or other motivational processes.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The fMRI slice tilt maximized ventromedial prefrontal signal but caused loss of coverage in dorso-posterior parietal areas including S1, so the contribution of S1 to pain avoidance learning could not be assessed.

“In order to maximize signal in the vmPFC, slices were tilted by 30° from AC-PC axis, resulting in a loss of coverage in dorso-posterior parietal areas, including S1 in the arm area. We were therefore unable to assess the contribution of S1 in pain avoidance learning.”

May combine two claims.

In the final dynamic causal model, ventromedial prefrontal cortex projects most directly to periaqueductal gray among value-encoding regions, and avoidance value is most closely related to putamen, which transmits value information to ventromedial prefrontal cortex.

“In the final, most likely model (Fig. 4), vmPFC projects most directly to PAG among value-encoding regions, and avoidance value is most closely related to the putamen, which transmits value information to vmPFC.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The ventral striatum region of interest was derived from the nucleus accumbens ROI reported by Rutledge et al. (2010).

“The VS ROI was based on Rutledge et al. (2010)9 nucleus accumbens ROI and was comprised of three 5 mm spheres for each hemisphere ([8 13 –3; 12 13 –8; 9 13 –7; –8 13 –3; –12 13 –8; –9 13 –7 ]).”

• cites Rutledge et al. (2010) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The DCM procedure cannot definitively establish causal relationships among regions, but the model offers a plausible working model given direct anatomical projections from vmPFC to PAG.

“Though this procedure cannot definitively isolate causal relationships among regions, this model provides a plausible working model considering the direct anatomical projections from vmPFC to PAG30.”

• cites [30] Price, J.L. Definition of the orbital cortex in relation to specific connections with limbic and visceral structures and other cortical regions. Ann. NY Acad. Sci. 1121, 54–71 (2007). sourceNot checked yet

  Not checked in the free run.

• AssumptionThe authors state this as an assumption: Primary nociceptive afferents project directly to the PAG.
• AssumptionThe authors state this as an assumption: Expected avoidance value reaches the PAG through at least one of the three expected-value structures (vmPFC, putamen, hippocampus).
• AssumptionThe authors state this as an assumption: Prediction error signals can be sent directly to the OFC and aMCC, and indirectly to the dmPFC via either aMCC or OFC.

Classified as probably not a substantial claim. May combine two claims.

According to reinforcement learning principles, PAG regions encoding aversive prediction errors should receive convergent input from expectancy-encoding regions (vmPFC, putamen, hippocampus) and primary reinforcement signals, whereas action-value and decision-making regions (aMCC, OFC, dmPFC) may receive convergent prediction error and primary reinforcement signals.

“Based on the principles governing RL models (Fig. 2a), regions that encode aversive PEs (PAG) should receive converging input from those that encode expectancies (vmPFC, putamen, hippocampus) and primary reinforcement (nociceptive) signals. Regions important for action value and decision-making (aMCC, OFC, and dmPFC) may receive converging PE and primary reinforcement signals.”

Probably not a substantial claim: may not pass S1 (asserted).

Animal models of fear conditioning suggest that regions encoding avoidance value receive prediction error signals generated in the PAG.

“Based on existent animal models of fear conditioning1,18, we posited that these regions receive the PE signals generated in PAG.”

• cites [1] McNally, G.P., Johansen, J.P. & Blair, H.T. Placing prediction into the fear circuit. Trends Neurosci. 34, 283–292 (2011). sourceNot checked yet

  Not checked in the free run.

• cites [18] Johansen, J.P., Tarpley, J.W., LeDoux, J.E. & Blair, H.T. Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray. Nat. Neurosci. 13, 979–986 (2010). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Axiom #3 could not be fully tested in Study 3 because cues never fully predicted outcomes and the temperature-pain relationship is non-linear.

“Unfortunately, this axiom cannot be fully tested here because the outcome is never fully predicted by the cue (50%–50%), and the non-linear relationship between temperature and pain makes it difficult to precisely estimate expectations.”