Reward-Guided Learning with and without Causal Attribution
Open accessAbstract
When an organism receives a reward, it is crucial to know which of many candidate actions caused this reward. However, recent work suggests that learning is possible even when this most fundamental assumption is not met. We used novel reward-guided learning paradigms in two fMRI studies to show that humans deploy separable learning mechanisms that operate in parallel. While behavior was dominated by precise contingent learning, it also revealed hallmarks of noncontingent learning strategies. These learning mechanisms were separable behaviorally and neurally. Lateral orbitofrontal cortex supported contingent learning and reflected contingencies between outcomes and their causal choices. Amygdala responses around reward times related to statistical patterns of learning. Time-based heuristic mechanisms were related to activity in sensorimotor corticostriatal circuitry. Our data point to the existence of several learning mechanisms in the human brain, of which only one relies on applying known rules about the causal structure of the task.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Area 47/12o has been linked to the process of assigning the credit for a specific outcome to a specific choice.
“It has been linked to the process of assigning the credit for a specific outcome to a specific choice.27”
From Two prefrontal-subcortical circuits estimate controllability and reflect its impact on learning (2026) · cites this paper as [27]
The Discussion links lateral orbitofrontal cortex (lOFC) activity to correctly assigning credit for contingent rewards, and the Results report lOFC responses aligned with choice–outcome contingencies. Our copy does not identify that region specifically as area 47/12o, so the claim’s anatomical specificity is not established. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Advances in functional and structural MR image analysis and implementation as FSL
Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., Bannister, P. R., De Luca, M., Drobnjak, I., Flitney, D. E., et al. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.051 · Cited by 13,630 · see citing works
Self-citationJenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002) · NeuroImage · doi:10.1006/nimg.2002.1132 · Cited by 11,888 · see citing works
A global optimisation method for robust affine registration of brain images
Jenkinson, M., & Smith, S. M. (2001) · Medical Image Analysis · doi:10.1016/s1361-8415(01)00036-6 · Cited by 6,806 · see citing works
Optimal foraging, the marginal value theorem
Charnov, E. L. (1976) · Theoretical Population Biology · doi:10.1016/0040-5809(76)90040-x · Cited by 5,374 · see citing works
Temporal Autocorrelation in Univariate Linear Modeling of FMRI Data
Woolrich, M. W., Ripley, B. D., Brady, M., & Smith, S. M. (2001) · NeuroImage · doi:10.1006/nimg.2001.0931 · Cited by 2,859 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402 · see citing works
KRINGELBACH, M. (2004) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2004.03.006 · Cited by 1,977 · see citing works
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works
Dissociation in prefrontal cortex of affective and attentional shifts
Dias, R., Robbins, T. W., & Roberts, A. C. (1996) · Nature · doi:10.1038/380069a0 · Cited by 1,527 · see citing works
Skinner, B. F. (1948) · Journal of Experimental Psychology · doi:10.1037/h0055873 · Cited by 1,456 · cited 1× · see citing works
“It also bears a resemblance to the emergence of superstitious behaviors during operant conditioning, where behaviors unrelated to reward are often reinforced due to their temporal proximity to reward delivery (Skinner, 1948; Devenport, 1979).”
BackgroundRelative reward preference in primate orbitofrontal cortex
Tremblay, L., & Schultz, W. (1999) · Nature · doi:10.1038/19525 · Cited by 1,314 · see citing works
Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey
Selemon, L. D., & Goldman-Rakic, P. S. (1985) · Journal of Neuroscience · doi:10.1523/jneurosci.05-03-00776.1985 · Cited by 1,074 · see citing works
Optimal decision making and the anterior cingulate cortex
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1724 · Cited by 948 · see citing works
Self-citationOrbitofrontal Cortex as a Cognitive Map of Task Space
Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.005 · Cited by 885 · see citing works
Limbic lesions and the problem of stimulus--reinforcement associations.
Jones, B., & Mishkin, M. (1972) · Experimental Neurology · doi:10.1016/0014-4886(72)90030-1 · Cited by 864 · see citing works
Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning
Schoenbaum, G. M., Chiba, A. A., & Gallagher, M. (1998) · Nature Neuroscience · doi:10.1038/407 · Cited by 849 · see citing works
The orbitofrontal cortex: Neuronal activity in the behaving monkey
Thorpe, S. J., Rolls, E. T., & Maddison, S. (1983) · Experimental Brain Research · doi:10.1007/bf00235545 · Cited by 836 · see citing works
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762 · see citing works
Fast, automated, N‐dimensional phase‐unwrapping algorithm
Jenkinson, M. (2002) · Magnetic Resonance in Medicine · doi:10.1002/mrm.10354 · Cited by 702 · see citing works
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 690 · see citing works
Frontal Granular Cortex and Behavior.
Orbach, J. (1964) · Archives of General Psychiatry · doi:10.1001/archpsyc.1964.01720300109016 · Cited by 627 · see citing works
A critical time window for dopamine actions on the structural plasticity of dendritic spines
Yagishita, S., Hayashi-Takagi, A., Ellis-Davies, G. C. R., Urakubo, H., Ishii, S., & Kasai, H. (2014) · Science · doi:10.1126/science.1255514 · Cited by 614 · see citing works
Izquierdo, A., Suda, R. K., & Murray, E. A. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.1921-04.2004 · Cited by 588 · see citing works
A new perspective on the role of the orbitofrontal cortex in adaptive behaviour
Schoenbaum, G., Roesch, M. R., Stalnaker, T. A., & Takahashi, Y. K. (2009) · Nature reviews. Neuroscience · doi:10.1038/nrn2753 · Cited by 585 · see citing works
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580 · see citing works
Dias, R., Robbins, T. W., & Roberts, A. C. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-23-09285.1997 · Cited by 555 · see citing works
Chudasama, Y., & Robbins, T. W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-25-08771.2003 · Cited by 543 · see citing works
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397 · see citing works
Self-citationThe representation of economic value in the orbitofrontal cortex is invariant for changes of menu
Padoa‐Schioppa, C., & Assad, J. A. (2007) · Nature Neuroscience · doi:10.1038/nn2020 · Cited by 390 · see citing works
Neural Evidence for Dissociable Components of Task-switching
Crone, E. A., Wendelken, C., Donohue, S. E., & Bunge, S. A. (2005) · Cerebral Cortex · doi:10.1093/cercor/bhi127 · Cited by 359 · see citing works
Frontal cortex subregions play distinct roles in choices between actions and stimuli
Rudebeck, P. H., Behrens, T. E., Kennerley, S. W., Baxter, M. G., Buckley, M. J., Walton, M. E., & Rushworth, M. F. S. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3541-08.2008 · Cited by 350 · see citing works
Self-citationExpectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280 · see citing works
The convergence of information about rewarding and aversive stimuli in single neurons
Morrison, S. E., & Salzman, C. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1815-09.2009 · Cited by 206 · see citing works
Dopaminergic Genes Predict Individual Differences in Susceptibility to Confirmation Bias
Doll, B. B., Hutchison, K. E., & Frank, M. J. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6486-10.2011 · Cited by 184 · see citing works
A mechanism for value-guided choice based on the excitation-inhibition balance in prefrontal cortex
Jocham, G., Hunt, L. T., Near, J., & Behrens, T. E. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3140 · Cited by 183 · see citing works
Self-citationModification of the Backlin-Kirk Combustion Chamber for Micro-Determination of Carbon and Lipoids
Doyle, W. L. (1933) · Science · doi:10.1126/science.77.1989.173-a · Cited by 181 · see citing works
Basolateral Amygdala Lesions Abolish Orbitofrontal-Dependent Reversal Impairments
Stalnaker, T. A., Franz, T. M., Singh, T., & Schoenbaum, G. M. (2007) · Neuron · doi:10.1016/j.neuron.2007.02.014 · Cited by 177 · see citing works
Rudebeck, P. H., & Murray, E. A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2272-08.2008 · Cited by 148 · see citing works
Dynamic Signals Related to Choices and Outcomes in the Dorsolateral Prefrontal Cortex
Seo, H., Barraclough, D. J., & Lee, D. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm064 · Cited by 143 · see citing works
The Role of Orbitofrontal Cortex in Decision Making
Fellows, L. K. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1401.023 · Cited by 133 · see citing works
Bandit problems and the exploration/exploitation tradeoff
Macready, W. G., & Wolpert, D. H. (1998) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.728210 · Cited by 120 · see citing works
Free-operant acquisition with delayed reinforcement
Dickinson, A., Watt, A. A., & Griffiths, W. J. (2007) · Quarterly Journal of Experimental Psychology · doi:10.1080/14640749208401019 · Cited by 99 · see citing works
Choice Coding in Frontal Cortex during Stimulus-Guided or Action-Guided Decision-Making
Luk, C.-H., & Wallis, J. D. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4920-12.2013 · Cited by 96 · see citing works
Monkey Orbitofrontal Cortex Encodes Response Choices Near Feedback Time
Tsujimoto, S., Genovesio, A., & Wise, S. P. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5777-08.2009 · Cited by 93 · see citing works
Encoding of Gustatory Working Memory by Orbitofrontal Neurons
Lara, A. H., Kennerley, S. W., & Wallis, J. D. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.4637-08.2009 · Cited by 76 · see citing works
Short-term memory traces for action bias in human reinforcement learning
Bogacz, R., McClure, S. M., Li, J., Cohen, J. D., & Montague, P. R. (2007) · Brain Research · doi:10.1016/j.brainres.2007.03.057 · Cited by 74 · see citing works
Trace conditioning of the conditioned emotional response.
Kamin, L. J. (1961) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0045611 · Cited by 56 · see citing works
Superstitious Bar Pressing in Hippocampal and Septal Rats
Devenport, L. D. (1979) · Science · doi:10.1126/science.462183 · Cited by 39 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior
Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.002 · Cited by 1,055 · see citing works
Self-citationMedial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244 · see citing works
Self-citationFouragnan, E., Retzler, C., & Philiastides, M. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24047 · Cited by 176 · see citing works
Specializations for reward-guided decision-making in the primate ventral prefrontal cortex
Murray, E. A., & Rudebeck, P. H. (2018) · Nature Reviews Neuroscience · doi:10.1038/s41583-018-0013-4 · Cited by 169 · see citing works
Noonan, M. P., Chau, B. K. H., Rushworth, M. F. S., & Fellows, L. K. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.0692-17.2017 · Cited by 167 · see citing works
Self-citationFolloni, D., Fouragnan, E., Wittmann, M. K., Roumazeilles, L., Tankelevitch, L., Verhagen, L., Attali, D., Aubry, J.-F., Sallet, J., & Rushworth, M. F. S. (2021) · Science Advances · doi:10.1126/sciadv.abg7700 · Cited by 75 · see citing works
Causal manipulation of self-other mergence in the dorsomedial prefrontal cortex
Wittmann, M. K., Trudel, N., Trier, H. A., Klein-Flügge, M. C., Sel, A., Verhagen, L., & Rushworth, M. F. S. (2021) · Neuron · doi:10.1016/j.neuron.2021.05.027 · Cited by 55 · see citing works
Self-citationMiyamoto, K., Trudel, N., Kamermans, K., Lim, M. C., Lazari, A., Verhagen, L., Wittmann, M. K., & Rushworth, M. F. S. (2021) · Neuron · doi:10.1016/j.neuron.2021.02.024 · Cited by 45 · see citing works
Self-citationImagining the future self through thought experiments
Miyamoto, K., Rushworth, M. F. S., & Shea, N. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.01.005 · Cited by 28 · see citing works
Decision-making as transdiagnostic construct for mental health research
Endrass, T., & Ullsperger, M. (2021) · Neuron · doi:10.1016/j.neuron.2021.05.035 · Cited by 7 · see citing works
Active disambiguation guides inferring controllability and cause in social interactions
Spiering, L., Trier, H. A., O’Reilly, J. X., Kolling, N., Wittmann, M. K., Rushworth, M. F. S., & Scholl, J. (2025) · Nature Communications · doi:10.1038/s41467-025-67853-8 · Cited by 2 · see citing works
Self-citationTwo prefrontal-subcortical circuits estimate controllability and reflect its impact on learning
Liu, Y., Spiering, L., Luo, S., Kingston, N., Grohn, J., & Rushworth, M. F. S. (2026) · Neuron · doi:10.1016/j.neuron.2026.07.029 · Cited by 1 · see citing works
Self-citation
- Authors
- Jocham, G., Brodersen, K. H., Constantinescu, A. O., Kahn, M. C., Ianni, A. M., Walton, M. E., Rushworth, M. F. S., & Behrens, T. E. J.
- Affiliations
- John Radcliffe Hospital (GB)
- National Institutes of Health (US)
- University of Oxford (GB)
- Wellcome Centre for Human Neuroimaging (GB)
- Corresponding author
- Jocham, Gerhard
- Brodersen, Kay H.
- ORCID
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2016-04
- Volume
- 90
- Issue
- 1
- Pages
- 177-190
- ISSN
- 0896-6273 (ISSN-L)
- PMC
- PMC4826429
- OpenAlex
- W2294594699
- Cited by
- 92
- References
- 51
- Citations by year
- 2026: 4 · 2025: 12 · 2024: 10 · 2023: 12 · 2022: 8 · 2021: 14 · 2020: 9 · 2019: 2 · 2018: 8 · 2017: 9 · 2016: 4
- FWCI
- 4.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Neuron (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- Wellcome Trust (104765/Z/14/Z, WT088312AIA, WT090051MA, WT104765MA, 099715/Z/12/Z, 100973/Z/13/Z)
- Economic and Social Research Council (1501067)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- causal attribution, lateral orbitofrontal cortex, amygdala, corticostriatal circuits, statistical learning, causal structure
- MeSH terms
- Heuristics, Adult, Amygdala, Brain, Brain Mapping, Cerebral Cortex, Choice Behavior, Female, Humans, Learning, Magnetic Resonance Imaging, Male, Mesencephalon, Neural Pathways, Reward, Prefrontal Cortex, Young Adult, Functional Neuroimaging, Ventral Striatum
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
