The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index.
Frontal theta links prediction errors to behavioral adaptation in reinforcement learning
Open accessAbstract
Investigations into action monitoring have consistently detailed a fronto-central voltage deflection in the Event-Related Potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the Feedback Related Negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single trial relationship between neural activity and the quanta of expectation violation remains untested. Although ERP methods are not well suited to single trial analyses, the FRN has been associated with theta band oscillatory perturbations in the medial prefrontal cortex. Medio-frontal theta oscillations have been previously associated with expectation violation and behavioral adaptation and are well suited to single trial analysis. Here, we recorded EEG activity during a probabilistic reinforcement learning task and fit the performance data to an abstract computational model (Q-learning) for calculation of single-trial reward prediction errors. Single-trial theta oscillatory activities following feedback were investigated within the context of expectation (prediction error) and adaptation (subsequent reaction time change). Results indicate that interactive medial and lateral frontal theta activities reflect the degree of negative and positive reward prediction error in the service of behavioral adaptation. These different brain areas use prediction error calculations for different behavioral adaptations: with medial frontal theta reflecting the utilization of prediction errors for reaction time slowing (specifically following errors), but lateral frontal theta reflecting prediction errors leading to working memory-related reaction time speeding for the correct choice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Neuronal Oscillations in Cortical Networks
Buzsáki, G., & Draguhn, A. (2004) · Science · doi:10.1126/science.1099745 · Cited by 6,576
Buzsáki, G. (2006) · doi:10.1093/acprof:oso/9780195301069.001.0001 · Cited by 4,873
A theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578
A mechanism for cognitive dynamics: neuronal communication through neuronal coherence.
Fries, P. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.08.011 · Cited by 4,284 · cited 1×
“…coherent oscillations are thought to reflect entrained inter-regional activity, increasing the coordination of spike timing across spatially separate neural networks and presumably reflecting functional communication (Buzsáki, 2006; Buzsaki and Draguhn, 2004; Fries, 2005; Womelsdorf et al., 2007).”
BackgroundHolroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907
Measuring phase synchrony in brain signals
Lachaux, J.-P., Rodriguez, E., Martinerie, J., & Varela, F. J. (1999) · Human Brain Mapping · doi:10.1002/(sici)1097-0193(1999)8:4<194::aid-hbm4>3.0.co;2-c · Cited by 3,798
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Influence of cognitive control and mismatch on the N2 component of the ERP: A review
Folstein, J. R., & Van Petten, C. (2008) · Psychophysiology · doi:10.1111/j.1469-8986.2007.00602.x · Cited by 2,567
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Self-citationThe Medial Frontal Cortex and the Rapid Processing of Monetary Gains and Losses
Gehring, W. J., & Willoughby, A. R. (2002) · Science · doi:10.1126/science.1066893 · Cited by 1,884
Primate anterior cingulate cortex: Where motor control, drive and cognition interface
Paus, T. (2001) · Nature Reviews Neuroscience · doi:10.1038/35077500 · Cited by 1,722 · cited 2×
“The ACC and surrounding mPFC may play a role in determining the value of exercised options based on environmental feedback (Glascher et al., 2009), which may be used to drive changes in behavior when external cues indicate that new strategies are required (Paus, 2001).”
A Parametric Study of Prefrontal Cortex Involvement in Human Working Memory
Braver, T. S., Cohen, J. D., Nystrom, L. E., Jonides, J., Smith, E. E., & Noll, D. C. (1997) · NeuroImage · doi:10.1006/nimg.1996.0247 · Cited by 1,694
Dynamic Brain Sources of Visual Evoked Responses
Makeig, S., Westerfield, M., Jung, T.-P., Enghoff, S., Townsend, J., Courchesne, E., & Sejnowski, T. J. (2002) · Science · doi:10.1126/science.1066168 · Cited by 1,630
Miltner, W. H. R., Braun, C. H., & Coles, M. G. H. (1997) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1997.9.6.788 · Cited by 1,530
Mining event-related brain dynamics
Makeig, S., Debener, S., Onton, J., & Delorme, A. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.03.008 · Cited by 1,493
Experience-weighted Attraction Learning in Normal Form Games
Camerer, C., & Hua Ho, T. (1999) · Econometrica · doi:10.1111/1468-0262.00054 · Cited by 1,487
Modulation of Neuronal Interactions Through Neuronal Synchronization
Womelsdorf, T., Schoffelen, J.-M., Oostenveld, R., Singer, W., Desimone, R., Engel, A. K., & Fries, P. (2007) · Science · doi:10.1126/science.1139597 · Cited by 1,367
Dorsal anterior cingulate cortex: A role in reward-based decision making
Bush, G., Vogt, B. A., Holmes, J., Dale, A. M., Greve, D., Jenike, M. A., & Rosen, B. R. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.012470999 · Cited by 1,096
Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., von Cramon, D. Y., & Engel, A. K. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.3286-05.2005 · Cited by 1,062
Independent Coding of Reward Magnitude and Valence in the Human Brain
Yeung, N., & Sanfey, A. G. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4537-03.2004 · Cited by 933
The N2 in go/no-go tasks reflects conflict monitoring not response inhibition
Donkers, F. C. L., & van Boxtel, G. J. M. (2004) · Brain and Cognition · doi:10.1016/j.bandc.2004.04.005 · Cited by 749
The feedback-related negativity reflects the binary evaluation of good versus bad outcomes
Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2006) · Biological Psychology · doi:10.1016/j.biopsycho.2005.04.001 · Cited by 699
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning
Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706111104 · Cited by 683
Self-citationCavanagh, J. F., Cohen, M. X., & Allen, J. J. B. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.4137-08.2009 · Cited by 669
Self-citationRole for Cingulate Motor Area Cells in Voluntary Movement Selection Based on Reward
Shima, K., & Tanji, J. (1998) · Science · doi:10.1126/science.282.5392.1335 · Cited by 650
Chapman, L. J., & Chapman, J. P. (1987) · Brain and Cognition · doi:10.1016/0278-2626(87)90118-7 · Cited by 641
Kayser, J., & Tenke, C. E. (2006) · Clinical Neurophysiology · doi:10.1016/j.clinph.2005.08.034 · Cited by 634
Reward expectation modulates feedback-related negativity and EEG spectra
Cohen, M. X., Elger, C. E., & Ranganath, C. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.11.056 · Cited by 586
Holroyd, C. B., Pakzad‐Vaezi, K. L., & Krigolson, O. E. (2008) · Psychophysiology · doi:10.1111/j.1469-8986.2008.00668.x · Cited by 582
Nieuwenhuis, S., Holroyd, C. B., Mol, N., & Coles, M. G. H. (2004) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2004.05.003 · Cited by 557
Luu, P., Tucker, D. M., & Makeig, S. (2004) · Clinical Neurophysiology · doi:10.1016/j.clinph.2004.03.031 · Cited by 545
ERP Correlates of Feedback and Reward Processing in the Presence and Absence of Response Choice
Yeung, N., Holroyd, C. B., & Cohen, J. D. (2005) · Cerebral Cortex · doi:10.1093/cercor/bhh153 · Cited by 520
Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2007) · Psychophysiology · doi:10.1111/j.1469-8986.2007.00567.x · Cited by 517
The Electrophysiological Dynamics of Interference during the Stroop Task
Hanslmayr, S., Pastötter, B., Bäuml, K.-H., Gruber, S., Wimber, M., & Klimesch, W. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20020 · Cited by 510
Electrophysiological Responses to Errors and Feedback in the Process of Action Regulation
Luu, P., Tucker, D. M., Derryberry, D., Reed, M., & Poulsen, C. (2003) · Psychological Science · doi:10.1111/1467-9280.01417 · Cited by 502
Modulation of activity in medial frontal and motor cortices during error observation
van Schie, H. T., Mars, R. B., Coles, M. G. H., & Bekkering, H. (2004) · Nature Neuroscience · doi:10.1038/nn1239 · Cited by 485
Brain potentials associated with expected and unexpected good and bad outcomes
Hajcak, G., Holroyd, C. B., Moser, J. S., & Simons, R. F. (2005) · Psychophysiology · doi:10.1111/j.1469-8986.2005.00278.x · Cited by 481
Medial prefrontal cell activity signaling prediction errors of action values
Matsumoto, M., Matsumoto, K., Abe, H., & Tanaka, K. (2007) · Nature Neuroscience · doi:10.1038/nn1890 · Cited by 437
Errors in reward prediction are reflected in the event-related brain potential
Holroyd, C. B., Nieuwenhuis, S., Yeung, N., & Cohen, J. D. (2003) · NeuroReport · doi:10.1097/00001756-200312190-00037 · Cited by 433
Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases
Frank, M. J., Woroch, B. S., & Curran, T. (2005) · Neuron · doi:10.1016/j.neuron.2005.06.020 · Cited by 426
Self-citationGläscher, J., Hampton, A. N., & O'Doherty, J. P. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn098 · Cited by 387
Strategic control and medial frontal negativity: Beyond errors and response conflict
Bartholow, B. D., Pearson, M. A., Dickter, C. L., Sher, K. J., Fabiani, M., & Gratton, G. (2005) · Psychophysiology · doi:10.1111/j.1469-8986.2005.00258.x · Cited by 387
Context dependence of the event‐related brain potential associated with reward and punishment
Holroyd, C. B., Larsen, J. T., & Cohen, J. D. (2004) · Psychophysiology · doi:10.1111/j.1469-8986.2004.00152.x · Cited by 379
Sauseng, P., Klimesch, W., Gruber, W. R., Hanslmayr, S., Freunberger, R., & Doppelmayr, M. (2007) · Neuroscience · doi:10.1016/j.neuroscience.2007.03.014 · Cited by 374
Nieuwenhuis, S., Ridderinkhof, K. R., Talsma, D., Coles, M. G. H., Holroyd, C. B., Kok, A., & van der Molen, M. W. (2002) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.2.1.19 · Cited by 348
Reward prediction error signals associated with a modified time estimation task
Holroyd, C. B., & Krigolson, O. E. (2007) · Psychophysiology · doi:10.1111/j.1469-8986.2007.00561.x · Cited by 341
Reinforcement Learning Signals Predict Future Decisions
Cohen, M. X., & Ranganath, C. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.4421-06.2007 · Cited by 333
Nieuwenhuis, S. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhh034 · Cited by 332
Nieuwenhuis, S., Slagter, H. A., Von Geusau, N. J. A., Heslenfeld, D. J., & Holroyd, C. B. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04152.x · Cited by 317
Potts, G. F., Martin, L. E., Burton, P., & Montague, P. R. (2006) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2006.18.7.1112 · Cited by 316
Theta EEG dynamics of the error-related negativity
Trujillo, L. T., & Allen, J. J. B. (2007) · Clinical Neurophysiology · doi:10.1016/j.clinph.2006.11.009 · Cited by 314 · cited 3×
“…and power enhancement over the medial frontal cortex (Bernat et al., 2008; Cavanagh et al., 2009; Cohen et al., 2007; Marco-Pallares et al., 2008; Trujillo and Allen, 2007), supporting the major postulate of Holroyd and Coles' (2002) reinforcement learning theory that these two ERPs reflect the…”
Self-citationDelayed feedback effects on rule-based and information-integration category learning.
Maddox, W. T., Ashby, F. G., & Bohil, C. J. (2003) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.29.4.650 · Cited by 313
Wang, C., Ulbert, I., Schomer, D. L., Marinkovic, K., & Halgren, E. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.4151-04.2005 · Cited by 295
Human oscillatory activity associated to reward processing in a gambling task
Marco-Pallares, J., Cucurell, D., Cunillera, T., García, R., Andrés-Pueyo, A., Münte, T. F., & Rodríguez-Fornells, A. (2008) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2007.07.016 · Cited by 272
Detection of synchronized oscillations in the electroencephalogram: An evaluation of methods
Yeung, N., Bogacz, R., Holroyd, C. B., & Cohen, J. D. (2004) · Psychophysiology · doi:10.1111/j.1469-8986.2004.00239.x · Cited by 250
Better or worse than expected? Aging, learning, and the ERN
Eppinger, B., Kray, J., Mock, B., & Mecklinger, A. (2008) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2007.09.001 · Cited by 248
Analysis of dynamic brain oscillations: methodological advances
Le Van Quyen, M., & Bragin, A. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.05.006 · Cited by 200
Cohen, M. X., Ridderinkhof, K. R., Haupt, S., Elger, C. E., & Fell, J. (2008) · Brain Research · doi:10.1016/j.brainres.2008.07.114 · Cited by 187
Theta phase resetting and the error‐related negativity
Yeung, N., Bogacz, R., Holroyd, C. B., Nieuwenhuis, S., & Cohen, J. D. (2007) · Psychophysiology · doi:10.1111/j.1469-8986.2006.00482.x · Cited by 185
Direct recording of Theta oscillations in primate prefrontal and anterior cingulate cortices
Tsujimoto, T., Shimazu, H., & Isomura, Y. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00730.2005 · Cited by 184
When is an error not a prediction error? An electrophysiological investigation
Holroyd, C. B., Krigolson, O. E., Baker, R., Lee, S., & Gibson, J. (2009) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.9.1.59 · Cited by 183
Neural Bases of Cognitive ERPs: More than Phase Reset
Fell, J., Dietl, T., Grunwald, T., Kurthen, M., Klaver, P., Trautner, P., Schaller, C., Elger, C. E., & Fernández, G. (2004) · Journal of Cognitive Neuroscience · doi:10.1162/0898929042568514 · Cited by 180
Error negativity on correct trials: a reexamination of available data
Vidal, F., Burle, B., Bonnet, M., Grapperon, J., & Hasbroucq, T. (2003) · Biological Psychology · doi:10.1016/s0301-0511(03)00097-8 · Cited by 177
Different neural systems adjust motor behavior in response to reward and punishment
Wrase, J., Kahnt, T., Schlagenhauf, F., Beck, A., Cohen, M. X., Knutson, B., & Heinz, A. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.04.001 · Cited by 172
van Veen, V., Holroyd, C. B., Cohen, J. D., Stenger, V. A., & Carter, C. S. (2004) · Brain and Cognition · doi:10.1016/j.bandc.2004.06.007 · Cited by 166
Blair, K., Marsh, A. A., Morton, J., Vythilingam, M., Jones, M., Mondillo, K., Pine, D. C., Drevets, W. C., & Blair, J. R. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1640-06.2006 · Cited by 165
Mediofrontal negativities in the absence of responding
Donkers, F. C. L., Nieuwenhuis, S., & Boxtel, G. J. M. V. (2005) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2005.09.007 · Cited by 165
Error-related negativity reflects detection of negative reward prediction error
Yasuda, A., Sato, A., Miyawaki, K., Kumano, H., & Kuboki, T. (2004) · Neuroreport · doi:10.1097/00001756-200411150-00027 · Cited by 154
Errors, Conflicts, and the Brain
Falkenstein, M. (2004) · Journal of Psychophysiology · doi:10.1027/0269-8803.18.4.153 · Cited by 145
Ridderinkhof, K. R., Nieuwenhuis, S., & Bashore, T. R. (2003) · Neuroscience Letters · doi:10.1016/s0304-3940(03)00566-4 · Cited by 108
Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex
Oya, H., Adolphs, R., Kawasaki, H., Bechara, A., Damásio, A. R., & Howard, M. A. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0500899102 · Cited by 98
A note on the oddball N200 and the feedback ERN
Holroyd, C. B. (2004) · Ghent University Academic Bibliography (Ghent University) · Cited by 97
Axiomatic methods, dopamine and reward prediction error
Caplin, A., & Dean, M. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.007 · Cited by 88
Frank, M. J., D’Lauro, C. J., & Curran, T. (2007) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.7.4.297 · Cited by 87
Self-citationCohen, M. X. (2006) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsl021 · Cited by 86
Bernat, E. M., Nelson, L. D., Holroyd, C. B., Gehring, W. J., & Patrick, C. J. (2008) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.801362 · Cited by 59
Detecting alpha rhythm phase reset by phase sorting: Caveats to consider
Ritter, P., & Becker, R. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.04.031 · Cited by 33
Cited by
Frontal theta as a mechanism for cognitive control
Cavanagh, J. F., & Frank, M. J. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.04.012 · Cited by 2,726
Self-citationTheta lingua franca: A common mid‐frontal substrate for action monitoring processes
Cavanagh, J. F., Zambrano‐Vazquez, L., & Allen, J. J. B. (2012) · Psychophysiology · doi:10.1111/j.1469-8986.2011.01293.x · Cited by 651
Self-citationSubthalamic nucleus stimulation reverses mediofrontal influence over decision threshold
Cavanagh, J. F., Wiecki, T. V., Cohen, M. X., Figueroa, C. M., Samanta, J., Sherman, S. J., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2925 · Cited by 645
Self-citationFrontal midline theta reflects anxiety and cognitive control: Meta-analytic evidence
Cavanagh, J. F., & Shackman, A. J. (2015) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2014.04.003 · Cited by 623
Self-citationWalsh, M. M., & Anderson, J. R. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.05.008 · Cited by 544
Metadata
- Authors
- Cavanagh, J. F., Frank, M. J., Klein, T. J., & Allen, J. J. B.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2010-02
- Volume
- 49
- Issue
- 4
- Pages
- 3198-3209
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2150643588
- Cited by
- 469
- References
- 82
- FWCI
- 7.31
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.