Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans
Open accessPublished version of a preprint
The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2 Single-trial discriminant analysis and EEG-informed fMRI regressors. (a) We used single-trial analysis of the EEG to perform binary discriminations between conditions of interest, here denoted as condition A and B (in red and blue respectively). We first estimated w, which is a linear weighting on the EEG sensor data (X) that maximally discriminates between the two conditions. This determines a task-related projection (y) of the data, in which the distance to the decision boundary reflects the decision certainty of the classifier in separating each of the relevant conditions. We treated the single-trial y amplitudes (single-trial variability [EEG STV]), as an index of how each condition of interest was encoded on individual trials. (b) Given these y values and their corresponding outcome-locked onset time points, we built fMRI regressors for subsequent GLM analyses. These regressors were all convolved with the canonical HRF. Details of specific events included in each EEG-informed fMRI regressor can be found in the main text (see fMRI analysis section).](https://figures.cochat.ai/paper-figures/52f2c22b2ce14b9d944e6eb215320858/spatiotemporal-neural-characterization-of-prediction-error-valence-and-surprise-figure-2.jpg)



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 3×
“Moreover, a significant interaction between RPE valence and surprise (χ(2)(1) = 21.”
InfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Advances in functional and structural MR image analysis and implementation as FSL
Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., Bannister, P. R., De Luca, M., Drobnjak, I., Flitney, D. E., et al. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.051 · Cited by 13,630
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 2×
“negative feedback trials (valence dimension) and (2) high vs.”
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487
Investigations into resting-state connectivity using independent component analysis
Beckmann, C. F., DeLuca, M., Devlin, J. T., & Smith, S. M. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1634 · Cited by 3,398
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244 · cited 1×
“Highly unpredicted outcomes (either positive or negative) can boost attention and facilitate learning, whereas less surprising outcomes attracting reduced attention might slow down the learning process instead 19, 20 .”
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · cited 2×
“In addition to these known reward structures, we also identified significant clusters in the lingual gyrus (LG), the middle temporal gyrus (MTG) and the precuneus, areas that have previously been implicated in memory retrieval and adaptive visual learning processes 39 .”
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · cited 1×
“σ β φ = − − P i v i v i ( ) ( ( ( ) ( )) ) (3) A A B”
Research works printed but not linked: 2
- Pearce, J. M. & Mackintosh, N. J. in Attention and Associative Learning: From Brain to Behaviour (eds Mitchell, C. & Le Pelley, M. E.) 11–39 (Oxford University Press, 2010).
- Duda, R. O., Hart, P. E. & Stork, D. G. Pattern Classification. (Wiley-Interscience, 2000).
Cited by
Beyond the FRN: Broadening the time-course of EEG and ERP components implicated in reward processing
Glazer, J. E., Kelley, N. J., Pornpattananangkul, N., Mittal, V. A., & Nusslock, R. (2018) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2018.02.002 · Cited by 361
Fouragnan, E. F., Chau, B. K. H., Folloni, D., Kolling, N., Verhagen, L., Klein-Flügge, M., Tankelevitch, L., Papageorgiou, G. K., Aubry, J.-F., Sallet, J., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0375-6 · Cited by 213
Self-citationPisauro, M. A., Fouragnan, E., Retzler, C., & Philiastides, M. G. (2017) · Nature Communications · doi:10.1038/ncomms15808 · Cited by 208
Self-citationFouragnan, E., Retzler, C., & Philiastides, M. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24047 · Cited by 176
“…identify potential regions integrating valence and surprise either into a signed RPE representation or a linear superposition of the two signals (Fouragnan et al., 2017), we performed a conjunction analysis between the valence (pattern A), the surprise (pattern B) and signed RPE (pattern C)…”
InfluentialSelf-citationHaarsma, J., Fletcher, P. C., Griffin, J. D., Taverne, H. J., Ziauddeen, H., Spencer, T. J., Miller, C., Katthagen, T., Goodyer, I., Diederen, K. M. J., et al. (2021) · Molecular Psychiatry · doi:10.1038/s41380-020-0803-8 · Cited by 108
“…and dACC is in line with earlier findings by Hayden (et al., 2011) who found unsigned prediction errors in the dACC of monkeys, and with prior fMRI studies in humans (Fletcher et al 2001, Turner et al 2004, Fouragnan et al., 2017; Fouragnan et al 2018; Metereau & Dreher 2012; Ide et al., 2013).”
Metadata
- Authors
- Fouragnan, E., Queirazza, F., Retzler, C., Mullinger, K. J., & Philiastides, M. G.
- Affiliations
- University of Oxford (GB)
- University of Glasgow (GB)
- University of Huddersfield (GB)
- University of Nottingham (GB)
- Corresponding author
- Fouragnan, Elsa
- Philiastides, Marios G.
- Venue
- Scientific Reports
- Published
- 2017-07-06
- Volume
- 7
- Issue
- 1
- Pages
- 4762-4762
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC5500565
- OpenAlex
- W2620599308
- Cited by
- 61
- References
- 92
- Citations by year
- 2026: 6 · 2025: 7 · 2024: 5 · 2023: 6 · 2022: 8 · 2021: 7 · 2020: 3 · 2019: 7 · 2018: 10 · 2017: 2
- FWCI
- 3.45
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- University of Huddersfield Repository (University of Huddersfield) (acceptedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- PEARL (University of Plymouth) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Nottingham ePrints (University of Nottingham) (submittedVersion, cc-by)
- Huddersfield Research Portal (University of Huddersfield) (submittedVersion, cc-by)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Funders
- Directorate for Biological Sciences
- Medical Research Council (MR/J000914/1, MR/N003403/1)
- Biotechnology and Biological Sciences Research Council (BB/J015393/2, BB/J015393/1, BB/J015393/1-2)
- Economic and Social Research Council (ES/L012995/1)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- reward prediction error, reward learning, probabilistic reversal learning, attentional networks, reward network, single-trial EEG, reinforcement learning
- MeSH terms
- Attention, Avoidance Learning, Brain, Brain Mapping, Choice Behavior, Electroencephalography, Female, Humans, Magnetic Resonance Imaging, Male, Nerve Net, Reinforcement, Verbal, Reward, Young Adult, Anticipation, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans. https://app.cochat.ai/papers/10.1038/s41598-017-04507-w
- The paper
- Fouragnan, E., Queirazza, F., Retzler, C., Mullinger, K. J., & Philiastides, M. G. (2017). Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans. Scientific Reports. https://doi.org/10.1038/s41598-017-04507-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.