Temporal difference models describe higher-order learning in humans
Open accessThe DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Abstract
The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“One study specifically showed that AIC encodes both prediction of pain and pain prediction error in the same task (Seymour et al., 2004), which directly supports the functional integration and predictive encoding account of AIC.”
From Anterior insular cortex and emotional awareness (2013) · cites this paper as Seymour et al., 2004
In the citing paper
“AIC responses encode both predicted pain and pain prediction error within a single task [33]”
From Interoceptive inference, emotion, and the embodied self (2013) · cites this paper as [33]
In the citing paper
“This phasic signal represents the difference between the predicted and the observed outcome, which is used to update the model such that future predictions are more accurate (3).”
From Aberrant Salience, Information Processing, and Dopaminergic Signaling in People at Clinical High Risk for Psychosis (2020) · cites this paper as [3]
In the citing paper
“Additionally, we hypothesized the anterior insular activity to be altered, because this region is crucially involved in RPE processing (Pessiglione et al., 2006; Seymour et al., 2004; Voon et al., 2010; Wittmann et al., 2008)”
From Cognitive flexibility in adolescence: Neural and behavioral mechanisms of reward prediction error processing in adaptive decision making during development (2015) · cites this paper as Seymour et al., 2004
In the citing paper
“It is well known that aIns activity often coincides with activation in the dmPFC (cf. Hauser et al., 2014a, 2014b; Nelson et al., 2010; Seymour et al., 2004).”
From Cognitive flexibility in adolescence: Neural and behavioral mechanisms of reward prediction error processing in adaptive decision making during development (2015) · cites this paper as Seymour et al., 2004
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“, which has close links with methods for prediction, and optimal action selection, in engineerin”
MethodReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationFunctional profiling of the Saccharomyces cerevisiae genome
Giaever, G., Chu, A. M., Ni, L., Connelly, C., Riles, L., Véronneau, S., Dow, S., Lucau-Danila, A., Anderson, K., André, B., et al. (2002) · Nature · doi:10.1038/nature00935 · Cited by 4,316
Systematic functional analysis of the Caenorhabditis elegans genome using RNAi
Kamath, R. S., Fraser, A. G., Dong, Y., Poulin, G., Durbin, R., Gotta, M., Kanapin, A., Le Bot, N., Moreno, S., Sohrmann, M., et al. (2003) · Nature · doi:10.1038/nature01278 · Cited by 3,616
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Self-citationAzmitia, E. C., & Segal, M. (1978) · Journal of Comparative Neurology · doi:10.1002/cne.901790311 · Cited by 1,909 · cited 1×
“) suggestion that dopamine neurons in the ventral tegmental area and in the substantia nigra report appetitive prediction error, it has been suggested that serotonin-producing neurons of the dorsal raphe nucleus, which send strong projections to the ventral striatu”
BackgroundAn opponent-process theory of motivation. I. Temporal dynamics of affect
Solomon, R. L., & Corbit, J. D. (1974) · Psychological Review · doi:10.1037/h0036128 · Cited by 1,704
Research works printed but not linked: 1
- Barto, A. G., Sutton, R. S. & Watkins, C. J. C. H. in Learning and Computational Neuroscience: Foundations of Adaptive Networks (eds Gabriel, M. & Moor, J.) 539–602 (MIT, Cambridge, Massachusetts, 1990)
Cited by
Items 501–600 of 605
Wimmer, G. E., & Büchel, C. (2015) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/035212 · Cited by 4
The control of tonic pain by active relief learningPreprint
Zhang, S., Mano, H., Lee, M., Yoshida, W., Kawato, M., Robbins, T. W., & Seymour, B. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/222653 · Cited by 3
Self-citationHamroun, S., Lebreton, M. P., & Palminteri, S. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/2uw85 · Cited by 3
The price of pain and the value of suffering
Vlaev, I., Seymour, B., Dolan, R. J., & Chater, N. (2008) · PsycEXTRA Dataset · doi:10.1037/e636742011-068 · Cited by 5
Self-citationCognitive and Neural Mechanisms of Motivation, Reward, and Self-Control
Goschke, T. (2025) · Springer eBooks · doi:10.1007/978-3-031-87947-0_10 · Cited by 2
Expectations and uncertainty shape pain perception during learning
Lacombe-Thibault, M., & Coll, M. (2025) · Journal of Pain · doi:10.1016/j.jpain.2025.105569 · Cited by 4
Pain and self-preservation in autonomous robots: From neurobiological models to psychiatric disease
Piccolo, L., Libera, F. D., Bonarini, A., Seymour, B., & Ishiguro, H. (2017) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2017.8329816 · Cited by 2
“[14], [15] and punishments [16] is well characterized by RL (temporal difference) models.”
BackgroundSelf-citationFiore, V. G., & Gu, X. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118821 · Cited by 5
Genetic, epigenetic, and environmental factors in serotonin-associated disease condition
Friedel, E., & Heinz, A. (2020) · Handbook of behavioral neuroscience · doi:10.1016/b978-0-444-64125-0.00047-5 · Cited by 4
Genetic, Epigenetic and Environmental Factors in Serotonin Associated Disease Condition
Friedel, E., & Heinz, A. (2010) · Handbook of behavioral neuroscience · doi:10.1016/s1569-7339(10)70108-9 · Cited by 3
RewardLearning: Contributions of Corticobasal Ganglia Circuits to Reward Value Signals
Fareri, D. S., & Delgado, M. R. (2013) · Cambridge University Press eBooks · doi:10.1017/cbo9780511843716.024 · Cited by 9
McClure, S. M., & D’Ardenne, K. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374620-7.00010-8 · Cited by 5
Neuro-imagerie des circuits cortico-sous corticaux
Lehéricy, S., Gérardin, E., & Delmaire, C. (2006) · L Encéphale · doi:10.1016/s0013-7006(06)78686-9 · Cited by 3
“Un algorithme dit de « différence temporelle » a permis de modéliser avec succès ce type de réponse dans le striatum ventral (53, 59, 60, 80).”
BackgroundPart 4: Reinforcement learning: Machine learning and natural learning
Ishii, S., & Yoshida, W. (2006) · New Generation Computing · doi:10.1007/bf03037338 · Cited by 3
Montague, P. R. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.07.005 · Cited by 3
Xu, X., Shen, D., Gao, Y., & Kai, W. A. N. G. (2012) · Acta Automatica Sinica · doi:10.3724/sp.j.1004.2012.00673 · Cited by 4
Learning and Representation of Declarative Memories by Single Neurons in the Human Brain
Rutishauser, U. (2008) · CaltechTHESIS (California Institute of Technology) · doi:10.7907/gx3n-qd05 · Cited by 3
Cerebellar involvement in learning to balance a cart-pole systemPreprint
Ludolph, N., Ernst, T., Müller, O., Goericke, S. L., Giese, M. A., Timmann, D., & Ilg, W. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/586990 · Cited by 3
Value generalization in human avoidance learningPreprint
Norbury, A., Robbins, T. W., & Seymour, B. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/223149 · Cited by 2
Self-citationAsymmetric representation of aversive prediction errors in Pavlovian threat conditioningPreprint
Ojala, K. E., Tzovara, A., Poser, B. A., Lutti, A., & Bach, D. R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.10.197665 · Cited by 2
“They have then regressed model-derived PEs onto blood-oxygen-dependent level (BOLD) signal and found correlations in striatum, a target region of reward PE-expressing midbrain neurons (Boll et al., 2013; Li et al., 2011; Seymour et al., 2004; Zhang et al., 2016), but also in the insula, periaqueductal grey, substantia”
BackgroundPark, J., Lee, S., Sul, S., & Chung, D. (2020) · doi:10.31234/osf.io/s4c79 · Cited by 2
BASAL GANGLIA PATHWAYS: BEYOND THE CLOSED-LOOP CIRCUITS WITH THE CEREBRAL CORTEX
Bostan, A. C. (2013) · D-Scholarship@Pitt (University of Pittsburgh) · Cited by 4
Validating a measure of the precision of ascending noxious signalsPreprint
Singh, A., Treister, R., Charalambous, C., Mancini, F., & Talmi, D. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/9s5gj · Cited by 2
Culbreth, A. J., Kasanova, Z., Ross, T. J., Salmeron, B. J., Gold, J. M., Stein, E. A., & Waltz, J. A. (2021) · Brain Sciences · doi:10.3390/brainsci11121610 · Cited by 1
Personalized information and willingness to pay for non-financial risk prevention: An experiment
Arrighi, Y., Crainich, D., Flambard, V., & Massin, S. (2022) · Journal of Risk and Uncertainty · doi:10.1007/s11166-022-09384-x · Cited by 1
“The thresholding procedure, designed to individually determine the stimulus while controlling for the heterogeneity of pain resistance among subjects, is derived from studies in health psychology using similar apparatus (e.g. Claes et al., 2016; Meulders et al., 2011; Peeters and Vlaeyen, 2011; Seymour et al. 2004; Vl…”
MethodThe Human Cerebellum Encodes Temporally Sensitive Reinforcement Learning Signals
Trach, J. E., Ou, Y., & McDougle, S. D. (2026) · Journal of Neuroscience · doi:10.1523/jneurosci.2313-25.2026 · Cited by 4
Computational parametric mapping of functional neuroimaging data
Steinkamp, S. R., Chaker, I., Hubert, F. M., Meder, D., & Hulme, O. J. (2026) · Imaging Neuroscience · doi:10.1162/imag.a.1130 · Cited by 1
“In some tasks, such as classical conditioning tasks, the learning rate parameters were fixed a priori by the modeler ( O’Doherty et al., 2003; Seymour et al., 2004).”
Culture as a Response to Uncertainty: Foundations of Computational Cultural Neuroscience
Tobler, P. N., & Christopoulos, G. I. (2016) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199357376.013.6 · Cited by 9
Calbert, G. (2006) · Chapman & Hall/CRC computer and information science series · doi:10.1201/9781420011012.ch3.4 · Cited by 1
Dynamic changes in feedback processing as learning progresses
김수현, Kang, E., & 김진희 (2015) · 한국심리학회지: 인지 및 생물 · doi:10.22172/cogbio.2015.27.3.005 · Cited by 1
Reward Processing in Motivational and Affective Disorders
Skandali, N. (2016) · Frontiers research topics · doi:10.3389/978-2-88919-986-0 · Cited by 1
Computational fingerprinting: a new approach to motivation deficit in neuropsychiatric diseases
Pessiglione, M. (2021) · Comptes Rendus Biologies · doi:10.5802/crbiol.61 · Cited by 1
Patt, V. M., Palombo, D. J., Esterman, M. S., & Verfaellie, M. H. (2022) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01873 · Cited by 1
“Other brain correlates of PERL+ included areas of dopamine projections in the ventral striatum, consistent with well-established literature (e.g., Jocham, Klein, & Ullsperger, 2011; den Ouden, Daunizeau, Roiser, Friston, & Stephan, 2010; Seymour et al., 2004; McClure, Berns, & Montague, 2003; O’Doherty, Dayan, Friston…”
Gibbons, H., Schmuck, J., & Schnuerch, R. (2023) · Biological Psychology · doi:10.1016/j.biopsycho.2023.108627 · Cited by 1
The spread of affective and semantic valence representations across states
Heimer, O., & Hertz, U. (2024) · Cognition · doi:10.1016/j.cognition.2023.105714 · Cited by 1
Pain and Value-Based Decision-Making: An Introduction and Implications for Neuroeconomics
López-Guzmán, S., & Atlas, L. Y. (2025) · Springer eBooks · doi:10.1007/978-3-032-02925-6_12 · Cited by 1
EFFECTS OF LOSS AVERSION ON THE EVALUATION OF DECISION OUTCOMES
Kokmotou, A. (2019) · University of Liverpool · doi:10.17638/03046759 · Cited by 1
“The neural representations of decision utility have been investigated by reinforcement learning models, which associate predictive cues with their subsequent outcomes, under the assumption that cues indicating higher rewards will be preferred in subsequent choices after learning has occurred (O’Doherty, 2004; Schultz,…”
BackgroundCaution in exploring the effects of distant past outcomes on sequential choices
Morita, K., & Nagase, A. M. (2019) · Neuroscience Research · doi:10.1016/j.neures.2019.12.011 · Cited by 1
Biomechanics as an Element of the Motion Clinimetry System
Dyszkiewicz, A. J., & Hruby, D. (2020) · IntechOpen eBooks · doi:10.5772/intechopen.92757 · Cited by 2
Human reinforcement learning: Insights from intracranial recordings and stimulation
Ramayya, A. G. (2014) · Scholarly Commons (University of Pennsylvania) · Cited by 1
Stem Cell Therapy for Heart Failure Using Cord Blood
Patel, A. N., Sakthivel, R., & Ichim, T. E. (2010) · Springer eBooks · doi:10.1007/978-1-84882-718-9_23 · Cited by 1
The economics, neurobiology and pharmacology of intertemporal choice in humans
Pine, A. (2011) · UCL Discovery (University College London) · Cited by 1
Brain, meaning, and computation
Klein, M. (2007) · OPUS Publication Server of the University of Stuttgart (University of Stuttgart) · doi:10.18419/opus-2608 · Cited by 2
Verguts, T., & Notebaert, W. (2008) · Cited by 1
Neural representation of social, monetary and chocolate reinforcer processing
Zhu, J. (2016) · Aston Publications Explorer (Aston University) · doi:10.48780/publications.aston.ac.uk.00030063 · Cited by 1
Winkelmeier, L., Filosa, C., Scheller, M. F., Hartig, R., Sack, M., Becker, R., Wolf, D., Reinwald, J., Gerchen, M. F., Sartorius, A., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.02.22.432268 · Cited by 1
A Behavioral Association Between Prediction Errors and Risk-Seeking: Theory and EvidencePreprint
Möller, M., Grohn, J., Manohar, S., & Bogacz, R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.04.29.067751 · Cited by 1
Ethical Artificial Intelligence
(2020) · doi:10.1287/lytx.2020.02.09 · Cited by 1
Miller, P. C. (2013) · Springer eBooks · doi:10.1007/978-1-4614-7320-6_312-3 · Cited by 2
Miller, P. C. (2015) · Springer eBooks · doi:10.1007/978-1-4614-6675-8_312 · Cited by 1
Miller, P. C. (2022) · Springer eBooks · doi:10.1007/978-1-0716-1006-0_312 · Cited by 1
Computational mechanisms underlying the dynamics of physical and cognitive fatiguePreprint
Matthews, J., Pisauro, M. A., Jurgelis, M., Mueller, T., Vassena, E., Chong, T. T. ‐., & Apps, M. A. J. (2023) · doi:10.31234/osf.io/n3mj2 · Cited by 1
Precision Functional Mapping of Imagined and Experienced PainPreprint
Sun, M. M. C., Petre, B., Bo, K., Bango, C. I., Hershkop, M., Shohan, S. L., Jung, H., & Wager, T. D. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.08.06.668906 · Cited by 1
“Pain is particularly crucial for both empathy 46 – 50 and personal decision-making and learning 51 , and it involves distinct pathways 52 , representations 53,54 and multivariate fMRI patterns 55,56 from other forms of somatosensation.”
The impact of motivational and affective context on error-induced learning
Unger, K. (2013) · Publications of the UdS (Saarland University) · doi:10.22028/d291-23358
“30 encode a PE signal in several studies using aversive learning procedures (Menon et al., 2007; Schiller et al., 2008; Seymour et al., 2004).”
BackgroundMethodPrior knowledge and present events in the brain
Carlsson, K. (2005) · Figshare
Caudate responses to reward and punishment are preserved in healthy older adults
Cox, K. M. (2007) · D-Scholarship@Pitt (University of Pittsburgh)
Prediction error dependent changes in brain connectivity during associative learning
den Ouden, H. E. M. (2009) · UCL Discovery (University College London) · Cited by 1
Cortical spatiotemporal plasticity in visual category learning
Xu, Y. (2013) · KiltHub Repository · doi:10.1184/r1/6715415 · Cited by 2
Chouard, T. (2004) · Nature · doi:10.1038/news040607-5
Aupperle, R. L. (2009) · KU ScholarWorks (The University of Kansas)
SerotoninSelectivelyModulatesRewardValueinHuman Decision-Making
Daw, N. D. (2012)
Complementary roles for amygdala and periaqueductal gray in temporal-difference fear learning
Cole, S., & McNally, G. P. (2009) · Learning & Memory · doi:10.1101/lm.1120509 · Cited by 42
“The present and other findings (Cicala et al. 1990; Fanselow 1998; Seymour et al. 2004; McNally and Cole 2006; Cole and McNally 2007b; Eippert et al. 2008) strongly suggest that the role of vlPAG is more complex than simplymediating CR expression.”
ResultThe Neural and Behavioral Processes Underlying Limb Selection
Elenzil, R. H. S. (2013) · eScholarship (California Digital Library)
An in vitro analysis of striatal microcircuitry
Elghaba, R. (2015) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York)
Investigating reward systems in the human midbrain with fMRI
Limbrick‐Oldfield, E. H. (2012) · Spiral (Imperial College London) · doi:10.25560/14511
Dayan, P., & Daw, N. D. (2008)
Self-citationSinger, H. S., Mink, J. W., Gilbert, D. L., & Jankovic, J. J. (2010) · Elsevier eBooks · doi:10.1016/b978-0-7506-9852-8.00006-0
The Effects of Atypical Antipsychotics on Implicit Reward Learning in Schizophrenia
Bradford, L. (2015) · QSpace (Queen's University Library) · Cited by 2
Computational modelling of the neural systems involved in schizophrenia
Thurnham, A. J. (2008) · University of Hertfordshire Research Archive (University of Hertfordshire) · Cited by 1
Davey, N., Done, D. J., FRANK, R. J., & Thurnham, A. J. (2006) · Proceedings of the Annual Meeting of the Cognitive Science Society
Emotion and predictive processing: emotions as perceptions?
Araya, J. M. (2018) · ERA
Increasing pool funds in public goods: The effects of deposit-based delayed rewards
Zhang, X., Chen, T., Chen, Q., & Xue-ya, L. (2020) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2020.110201
Primate anterior insular cortex represents economic decision variables postulated by Prospect theoryPreprint
Yang, Y.-P., Li, X., & Stuphorn, V. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.11.12.380758
Encoding Motivation Prediction Errors in the Human Dopaminergic Reward SystemPreprint
Ni, Y., Wang, S., Su, J., Li, J., & Wan, X. (2020) · Research Square · doi:10.21203/rs.3.rs-51287/v1
Breaking the Cycle of Learned Fear: An Experimental Approach
van Dis, E. A. M. (2021) · Utrecht University Repository (Utrecht University) · doi:10.33540/934
Vitartas, P., Sloan, K., Poh, P., & Dunlop, M. (2001) · Cross Connect (Southern Cross University) · Cited by 1
Holcomb, H. H., & Rowland, L. M. (2007)
The role of cerebellum in action acquisition and motor learning
Dagioglou, M. (2014) · University of Birmingham Institutional Research Archive (University of Birmingham)
Effort-based learning and decision making
Kurniawan, I. T. (2011) · UCL Discovery (University College London)
Reward-Punishment Processing and Learning
Kim, H. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-819641-0.00150-x
Schnitzler, A., & Pollok, B. (2010) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.2768957.2434055
An Earth Observation Satellite Mission Planning Method Based on Deep Q-Learning
Chen, T., Wang, H., Liu, H., & Wu, P. (2021) · International Symposium on Computer and Information Processing Technology (ISCIPT) · doi:10.1109/iscipt53667.2021.00161
Dhamija, P., Allison Ka Yee, W. O. N. G., & Gilboa, A. (2022) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2022.751274
SUZUKI, W. A. (2007) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.1092002.545356
Simonyan, K. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.733028392.793555306
Fellows, L. K. (2020) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.727222049.793571330
Hsu, M., & Preuschoff, K. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-397025-1.00260-8
Nyberg, L. H. (2018) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.733028392.793546055
How instructions, learning, and expectations shape pain and clinical outcomesPreprint
Atlas, L. Y. (2022) · doi:10.31234/osf.io/qhkat
Mental Effort Cost Learning is RetrospectivePreprint
Nagase, A. M., Akaishi, R., Onoda, K., Morita, K., Kawagoe, T., Yamamoto, T., Yamaguchi, S., Hanajima, R., & Westbrook, A. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.11.26.518013 · Cited by 2
Collins, A. G. E. (2010) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 5
Validating a measure of the precision of ascending signals in pain perception
Singh, A., Treister, R., Charalambous, C., Mancini, F., & Talmi, D. (2025) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218251343863 · Cited by 3
“The electrical stimuli are harmless and have been used for many years in behavioural/ neuroimaging studies of sensation, pain and aversive learning (Norbury et al., 2018; Seymour et al., 2004; Vlaev et al., 2009).”
Data-driven equation discovery reveals nonlinear reinforcement learning in humans
LaFollette, K. J., Yuval, J., Schurr, R., Melnikoff, D. E., & Goldenberg, A. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2413441122 · Cited by 6
“…Q - Weighted model provides a unique perspective on probability weighting, it departs from the broader theoretical basis of RL models like Temporal Difference (TD) learning ( 2 ), which can be applied in real - time and have demonstrated strong links to learning processes in the brain ( 48 , 65 ).”
Kognitive und neuronale Grundlagen von Motivation, Belohnung und Selbstkontrolle
Goschke, T. (2025) · Springer eBooks · doi:10.1007/978-3-662-69368-1_10
Dynamic learning of the meaning of information changes pain perception
Hird, E., Charalambous, C., El‐Deredy, W., Jones, A. K. P., & Talmi, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-14299-z · Cited by 2
Self-citationMahajan, P., & Seymour, B. (2025) · Pain · doi:10.1097/j.pain.0000000000003705 · Cited by 2
“This approach was soon extended to explain simple predictive pain learning [43], aligning with fear and aversive conditioning [23], and affective-motivational theories of pain [14,15].”
Self-citationPrediction Errors in Neural Processing: Imaging in Humans
Lohrenz, T. M., & Montague, P. R. (2009) · Elsevier eBooks · doi:10.1016/b978-008045046-9.01556-4
The Brain Valuation System and its role in decision-making
Lopez‐Persem, A. (2016) · theses.fr (ABES) · doi:10.70675/d4ee5e83z099dz4d34z89ddzf182df6a9cf2 · Cited by 3
Order matters : the effect of sequential presentation on economic decision making
Hu, C. (2019) · theses.fr (ABES) · doi:10.70675/c9a41438z9143z4fa3z83ffz28b6952f6851
Cecchi, R. (2021) · theses.fr (ABES) · doi:10.70675/db89692cz3297z43beza1dez9029c96a67d3 · Cited by 1
Metadata
- Authors
- Seymour, B., O'Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S.
- Affiliations
- Wellcome Centre for Human Neuroimaging (GB)
- Gatsby Computational Neuroscience Unit (GB)
- University College London (GB)
- University of Manchester (GB)
- Corresponding author
- Seymour, Ben
- ORCID
- Venue
- Nature
- Published
- 2004-06
- Volume
- 429
- Issue
- 6992
- Pages
- 664-667
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2084912121
- Cited by
- 605
- References
- 60
- Citations by year
- 2026: 8 · 2025: 15 · 2024: 8 · 2023: 16 · 2022: 21 · 2021: 17 · 2020: 22 · 2019: 25 · 2018: 24 · 2017: 21 · 2016: 24 · 2015: 31 · 2014: 32 · 2013: 47 · 2012: 37
- FWCI
- 15.70
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- other-oa
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Max Planck Digital Library (submittedVersion, other-oa)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- temporal difference learning, higher-order conditioning, classical conditioning, aversive conditioning, pain prediction, ventral striatum, anterior insula, sequential learning, computational models of learning, reward learning
- MeSH terms
- Conditioning, Classical, Cues, Electric Stimulation, Hand, Humans, Learning, Magnetic Resonance Imaging, Models, Neurological, Pain, Punishment, Time Factors, Neostriatum
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.