The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index.
Frontal theta links prediction errors to behavioral adaptation in reinforcement learning
Open accessAbstract
Investigations into action monitoring have consistently detailed a fronto-central voltage deflection in the Event-Related Potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the Feedback Related Negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single trial relationship between neural activity and the quanta of expectation violation remains untested. Although ERP methods are not well suited to single trial analyses, the FRN has been associated with theta band oscillatory perturbations in the medial prefrontal cortex. Medio-frontal theta oscillations have been previously associated with expectation violation and behavioral adaptation and are well suited to single trial analysis. Here, we recorded EEG activity during a probabilistic reinforcement learning task and fit the performance data to an abstract computational model (Q-learning) for calculation of single-trial reward prediction errors. Single-trial theta oscillatory activities following feedback were investigated within the context of expectation (prediction error) and adaptation (subsequent reaction time change). Results indicate that interactive medial and lateral frontal theta activities reflect the degree of negative and positive reward prediction error in the service of behavioral adaptation. These different brain areas use prediction error calculations for different behavioral adaptations: with medial frontal theta reflecting the utilization of prediction errors for reaction time slowing (specifically following errors), but lateral frontal theta reflecting prediction errors leading to working memory-related reaction time speeding for the correct choice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Neuronal Oscillations in Cortical Networks
Buzsáki, G., & Draguhn, A. (2004) · Science · doi:10.1126/science.1099745 · Cited by 6,576
Buzsáki, G. (2006) · doi:10.1093/acprof:oso/9780195301069.001.0001 · Cited by 4,873
A theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578
A mechanism for cognitive dynamics: neuronal communication through neuronal coherence.
Fries, P. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.08.011 · Cited by 4,284 · cited 1×
“…coherent oscillations are thought to reflect entrained inter-regional activity, increasing the coordination of spike timing across spatially separate neural networks and presumably reflecting functional communication (Buzsáki, 2006; Buzsaki and Draguhn, 2004; Fries, 2005; Womelsdorf et al., 2007).”
BackgroundHolroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907
Cited by
Frontal theta as a mechanism for cognitive control
Cavanagh, J. F., & Frank, M. J. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.04.012 · Cited by 2,726
Self-citationTheta lingua franca: A common mid‐frontal substrate for action monitoring processes
Cavanagh, J. F., Zambrano‐Vazquez, L., & Allen, J. J. B. (2012) · Psychophysiology · doi:10.1111/j.1469-8986.2011.01293.x · Cited by 651
Self-citationSubthalamic nucleus stimulation reverses mediofrontal influence over decision threshold
Cavanagh, J. F., Wiecki, T. V., Cohen, M. X., Figueroa, C. M., Samanta, J., Sherman, S. J., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2925 · Cited by 645
Self-citationFrontal midline theta reflects anxiety and cognitive control: Meta-analytic evidence
Cavanagh, J. F., & Shackman, A. J. (2015) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2014.04.003 · Cited by 623
Self-citationWalsh, M. M., & Anderson, J. R. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.05.008 · Cited by 544
Frontal theta activity reflects distinct aspects of mental fatigue
Wascher, E., Rasch, B., Sänger, J., Hoffmann, S., Schneider, D., Rinkenauer, G., Heuer, H., & Gutberlet, I. (2014) · Biological Psychology · doi:10.1016/j.biopsycho.2013.11.010 · Cited by 454
Collins, A. G. E., & Frank, M. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07980.x · Cited by 417
Self-citationBeyond the FRN: Broadening the time-course of EEG and ERP components implicated in reward processing
Glazer, J. E., Kelley, N. J., Pornpattananangkul, N., Mittal, V. A., & Nusslock, R. (2018) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2018.02.002 · Cited by 361
Single-Trial Regression Elucidates the Role of Prefrontal Theta Oscillations in Response Conflict
Cohen, M. X., & Cavanagh, J. F. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00030 · Cited by 343
Self-citationFrank, M. J., & Badre, D. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr114 · Cited by 337
Self-citationBoksem, M. A. S., & Smidts, A. (2015) · Journal of Marketing Research · doi:10.1509/jmr.13.0572 · Cited by 304
Frontal Theta Reflects Uncertainty and Unexpectedness during Exploration and Exploitation
Cavanagh, J. F., Figueroa, C. M., Cohen, M. X., & Frank, M. J. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr332 · Cited by 291
Self-citationDecision processes in human performance monitoring
Steinhauser, M., & Yeung, N. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1899-10.2010 · Cited by 284
“…stages of error processing implemented in specific networks previously shown to be differentially active for detected and undetected errors (Klein et al., 2007; Ullsperger et al., 2010), or using formal models to guide single-trial analysis (Cavanagh et al., 2010; Philiastides et al., 2010).”
Gruzelier, J. H. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.03.015 · Cited by 270
van Driel, J., Ridderinkhof, K. R., & Cohen, M. X. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0802-12.2012 · Cited by 225
Dopaminergic Genes Predict Individual Differences in Susceptibility to Confirmation Bias
Doll, B. B., Hutchison, K. E., & Frank, M. J. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6486-10.2011 · Cited by 184
Self-citationHuman Scalp Electroencephalography Reveals that Repetition Suppression Varies with Expectation
Summerfield, C., Wyart, V., Johnen, V. M., & de Gardelle, V. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00067 · Cited by 145
Alpha and theta band dynamics related to sentential constraint and word expectancy
Rommers, J., Dickson, D. S., Norton, J. J. S., Wlotko, E. W., & Federmeier, K. D. (2017) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2016.1183799 · Cited by 109
Characterizing the roles of alpha and theta oscillations in multisensory attention
Keller, A. S., Payne, L., & Sekuler, R. (2017) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.02.021 · Cited by 106
McLoughlin, G., Palmer, J. A., Rijsdijk, F., & Makeig, S. (2014) · Biological Psychiatry · doi:10.1016/j.biopsych.2013.07.020 · Cited by 100
Wischnewski, M., Zerr, P., & Schutter, D. J. L. G. (2016) · Brain Stimulation · doi:10.1016/j.brs.2016.04.011 · Cited by 62
Interregional alpha-band synchrony supports temporal cross-modal integration
van Driel, J., Knapen, T., van Es, D. M., & Cohen, M. X. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.07.022 · Cited by 66
“This method has previously been applied to sharpening EEG topography and performing synchronization analyses (Cavanagh et al., 2010; van Driel et al., 2012).”
MethodBellato, A., Norman, L., Idrees, I., Ogawa, C. Y., Waitt, A., Zuccolo, P. F., Tye, C., Radua, J., Groom, M. J., & Shephard, E. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.10.018 · Cited by 49
Reward feedback processing in children and adolescents: medial frontal theta oscillations
Crowley, M. J., van Noordt, S. J. R., Wu, J., Hommer, R. E., South, M., Fearon, R. M. P., & Mayes, L. C. (2014) · Brain and Cognition · doi:10.1016/j.bandc.2013.11.011 · Cited by 45
“Future work will need to link developmental patterns of theta oscillation to learning (Cavanagh et al. 2010), decision making (Cohen et al., 2009) and risk behavior inside and outside the laboratory (Lejuez, Aklin, Zvolensky, & Pedulla, 2003).”
BackgroundInfluentialBrain oscillations and predictive processing.
Yordanova, J., Kolev, V., & Kirov, R. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00416 · Cited by 15
Metadata
- Authors
- Cavanagh, J. F., Frank, M. J., Klein, T. J., & Allen, J. J. B.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2010-02
- Volume
- 49
- Issue
- 4
- Pages
- 3198-3209
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2150643588
- Cited by
- 469
- References
- 82
- FWCI
- 7.31
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.