The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Model‐Based fMRI and Its Application to Reward Learning and Decision Making
Abstract
In model‐based functional magnetic resonance imaging (fMRI), signals derived from a computational model for a specific cognitive process are correlated against fMRI data from subjects performing a relevant task to determine brain regions showing a response profile consistent with that model. A key advantage of this technique over more conventional neuroimaging approaches is that model‐based fMRI can provide insights into how a particular cognitive process is implemented in a specific brain area as opposed to merely identifying where a particular process is located. This review will briefly summarize the approach of model‐based fMRI, with reference to the field of reward learning and decision making, where computational models have been used to probe the neural mechanisms underlying learning of reward associations, modifying action choice to obtain reward, as well as in encoding expected value signals that reflect the abstract structure of a decision problem. Finally, some of the limitations of this approach will be discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Statistical parametric maps in functional imaging: A general linear approach
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. ‐., Frith, C. D., & Frackowiak, R. S. J. (1994) · Human Brain Mapping · doi:10.1002/hbm.460020402 · Cited by 9,688
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 9,462
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Self-citationDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citation
Research works printed but not linked: 1
- Houk J.C., 1995, Models of Information Processing in the Basal Ganglia, 249
Cited by
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Neurocognitive aspects of pain perception
Wiech, K., Ploner, M., & Tracey, I. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.05.005 · Cited by 737
Ten simple rules for the computational modeling of behavioral data
Wilson, R. C., & Collins, A. G. (2019) · eLife · doi:10.7554/elife.49547 · Cited by 710
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 704
Uncertainty in perception and the Hierarchical Gaussian Filter
Mathys, C. D., Lomakina, E. I., Daunizeau, J., Iglesias, S., Brodersen, K. H., Friston, K. J., & Stephan, K. E. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00825 · Cited by 465
The ecology of human fear: survival optimization and the nervous system
Mobbs, D., Hagan, C. C., Dalgleish, T., Silston, B., & Prévost, C. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00055 · Cited by 498
Modalities, Modes, and Models in Functional Neuroimaging
Friston, K. J. (2009) · Science · doi:10.1126/science.1174521 · Cited by 489
How the brain integrates costs and benefits during decision making
Basten, U., Biele, G., Heekeren, H. R., & Fiebach, C. J. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0908104107 · Cited by 466
Trial-by-trial data analysis using computational models
Daw, N. D. (2011) · Decision Making, Affect, and Learning · doi:10.1093/acprof:oso/9780199600434.003.0001 · Cited by 456
An automatic valuation system in the human brain: Evidence from functional neuroimaging
Lebreton, M., Jorge, S., Michel, V., Thirion, B., & Pessiglione, M. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.040 · Cited by 455
“Considerable evidence from both human and nonhuman species supports the idea that the limbic circuit is involved in reward processing (Daw and Doya, 2006; Knutson and Cooper, 2005; McClure et al., 2004c; O'Doherty et al., 2007; Padoa-Schioppa, 2007; Rushworth and Behrens , 2008).”
The Importance of Falsification in Computational Cognitive Modeling
Palminteri, S., Wyart, V., & Koechlin, E. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.03.011 · Cited by 448
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 438
Oldham, S., Murawski, C., Fornito, A., Youssef, G., Yücel, M., & Lorenzetti, V. (2018) · Human Brain Mapping · doi:10.1002/hbm.24184 · Cited by 425
Dopaminergic dysfunction in schizophrenia: Salience attribution revisited
Heinz, A., & Schlagenhauf, F. (2010) · Schizophrenia Bulletin · doi:10.1093/schbul/sbq031 · Cited by 412
Collins, A. G. E., & Frank, M. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07980.x · Cited by 417
Neural mechanisms of observational learning
Burke, C. J., Tobler, P. N., Baddeley, M., & Schultz, W. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1003111107 · Cited by 367
Ahn, W.-Y., Haines, N., & Zhang, L. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00002 · Cited by 362
Leotti, L. A., & Delgado, M. R. (2011) · Psychological Science · doi:10.1177/0956797611417005 · Cited by 318
Subliminal Instrumental Conditioning Demonstrated in the Human Brain
Pessiglione, M., Petrovic, P., Daunizeau, J., Palminteri, S., Dolan, R. J., & Frith, C. D. (2008) · Neuron · doi:10.1016/j.neuron.2008.07.005 · Cited by 308
Neural mechanisms underlying motivation of mental versus physical effort
Schmidt, L., Lebreton, M., Cléry-Melin, M.-L., Daunizeau, J., & Pessiglione, M. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001266 · Cited by 310
Goschke, T. (2014) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1410 · Cited by 293
Behavioural and neural characterization of optimistic reinforcement learning
Lefebvre, G., Lebreton, M., Meyniel, F., Bourgeois-Gironde, S., & Palminteri, S. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0067 · Cited by 290
Contextual modulation of value signals in reward and punishment learning
Palminteri, S., Khamassi, M., Joffily, M., & Coricelli, G. (2015) · Nature Communications · doi:10.1038/ncomms9096 · Cited by 286
Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia
Ito, M., & Doya, K. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.6157-08.2009 · Cited by 286
The neural and computational systems of social learning
Olsson, A., Knapska, E., & Lindström, B. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0276-4 · Cited by 272
Prestimulus alpha and mu activity predicts failure to inhibit motor responses
Mazaheri, A., Nieuwenhuis, I. L. C., van Dijk, H., & Jensen, O. (2009) · Human Brain Mapping · doi:10.1002/hbm.20763 · Cited by 262
Serotonin Selectively Modulates Reward Value in Human Decision-Making
Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0053-12.2012 · Cited by 252
Representation of aversive prediction errors in the human periaqueductal gray
Roy, M., Shohamy, D., Daw, N., Jepma, M., Wimmer, G. E., & Wager, T. D. (2014) · Nature Neuroscience · doi:10.1038/nn.3832 · Cited by 246
Neural systems underlying approach and avoidance in anxiety disorders
Robin L., A., & Martin, P. P. (2010) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2010.12.4/raupperle · Cited by 227
Reinforcement learning in depression: a review of computational research
Chen, C., Takahashi, T., Nakagawa, S., Inoue, T., & Kusumi, I. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.05.005 · Cited by 233
Dynamic updating of hippocampal object representations reflects new conceptual knowledge
Mack, M. L., Love, B. C., & Preston, A. R. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1614048113 · Cited by 223
Fouragnan, E., Retzler, C., & Philiastides, M. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24047 · Cited by 176
The development of the ventral prefrontal cortex and social flexibility
Nelson, E. E., & Guyer, A. E. (2011) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2011.01.002 · Cited by 201
Yoshida, W., Seymour, B., Koltzenburg, M., & Dolan, R. J. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4984-12.2013 · Cited by 165
Assessing inter individual differences with task related functional neuroimaging
Lebreton, M., Bavard, S., Daunizeau, J., & Palminteri, S. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0681-8 · Cited by 160
Compulsivity reveals a novel dissociation between action and confidence
Vaghi, M. M., Luyckx, F., Sule, A., Fineberg, N. A., Robbins, T. W., & De Martino, B. (2017) · Neuron · doi:10.1016/j.neuron.2017.09.006 · Cited by 145
Effects of Direct Social Experience on Trust Decisions and Neural Reward Circuitry
Fareri, D. S., Chang, L. J., & Delgado, M. R. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00148 · Cited by 147
“…social decision-making (Chang et al., 2010) and has been combined with fMRI to highlight the neural processes associated with these computational processes (O’Doherty et al., 2007; Behrens et al., 2008; Hampton et al., 2008; Chang and Sanfey, 2011; Chang et al., 2011; Jones et al., 2011).”
BackgroundA predictive coding account of bistable perception - a model-based fMRI study
Weilnhammer, V., Stuke, H., Hesselmann, G., Sterzer, P., & Schmack, K. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005536 · Cited by 120
Chrastil, E. R., Sherrill, K. R., Hasselmo, M. E., & Stern, C. E. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.1209-15.2015 · Cited by 128
Computational and dynamic models in neuroimaging
Friston, K. J., & Dolan, R. J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.068 · Cited by 131
Zhu, L., Mathewson, K. E., & Hsu, M. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1116783109 · Cited by 125
Generalization of value in reinforcement learning by humans
Wimmer, G. E., Daw, N. D., & Shohamy, D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08017.x · Cited by 111
Model-based approaches to neuroimaging: combining reinforcement learning theory with fMRI data
Gläscher, J. P., & O'Doherty, J. P. (2010) · WIREs Cognitive Science · doi:10.1002/wcs.57 · Cited by 104
Self-citationSolving the detour problem in navigation: a model of prefrontal and hippocampal interactions
Spiers, H. J., & Gilbert, S. J. (2015) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2015.00125 · Cited by 94
The Algorithmic Level Is the Bridge Between Computation and Brain
Love, B. C. (2015) · Topics in Cognitive Science · doi:10.1111/tops.12131 · Cited by 82
Evidence for model-based computations in the human amygdala during Pavlovian conditioning
Prévost, C., McNamee, D., Jessup, R. K., Bossaerts, P., & O'Doherty, J. P. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002918 · Cited by 79
“Whereas the model-free system has received considerable attention in the past [26], the more sophisticated and flexible model-based system, has been more sparsely studied particularly in relation to its role in Pavlovian learning.”
BackgroundStriatal dopamine, reward, and decision making in schizophrenia
Deserno, L., Schlagenhauf, F., & Heinz, A. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/ldeserno · Cited by 58
Borst, J. P., Taatgen, N. A., & van Rijn, H. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.05.084 · Cited by 33
Active inference as a computational framework for consciousness
Vilas, M. G., Auksztulewicz, R., & Melloni, L. (2022) · Review of Philosophy and Psychology · doi:10.1007/s13164-021-00579-w · Cited by 28
Abnormal reward valuation and event-related connectivity in unmedicated major depressive disorder
Rupprechter, S., Stankevicius, A., Huys, Q. J. M., Series, P., & Steele, J. D. (2021) · Psychological Medicine · doi:10.1017/s0033291719003799 · Cited by 27
Gruszka, A., Hampshire, A., Barker, R. A., & Owen, A. M. (2017) · Cortex · doi:10.1016/j.cortex.2017.05.020 · Cited by 25
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
Self-citationAddressing altered anticipation as a transdiagnostic target through computational psychiatry
Sepúlveda, P., Aitsahalia, I., Kumar, K., Atkin, T., & Iigaya, K. (2025) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2025.02.014 · Cited by 3
Metadata
- Authors
- O'DOHERTY, J. P., HAMPTON, A., & KIM, H.
- Venue
- Annals of the New York Academy of Sciences
- Publisher
- Wiley
- Published
- 2007-05
- Volume
- 1104
- Issue
- 1
- Pages
- 35-53
- ISSN
- 0077-8923 (ISSN-L)
- 1749-6632
- OpenAlex
- W2090801405
- Cited by
- 508
- References
- 37
- FWCI
- 8.04
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.