Neural computations underlying action-based decision making in the human brain
Open accessThe DOI matches its Crossref record; 105 of the 106 references it cites were found in a registry or an index.
Abstract
Action-based decision making involves choices between different physical actions to obtain rewards. To make such decisions the brain needs to assign a value to each action and then compare them to make a choice. Using fMRI in human subjects, we found evidence for action-value signals in supplementary motor cortex. Separate brain regions, most prominently ventromedial prefrontal cortex, were involved in encoding the expected value of the action that was ultimately taken. These findings differentiate two main forms of value signals in the human brain: those relating to the value of each available action, likely reflecting signals that are a precursor of choice, and those corresponding to the expected value of the action that is subsequently chosen, and therefore reflecting the consequence of the decision process. Furthermore, we also found signals in the dorsomedial frontal cortex that resemble the output of a decision comparator, which implicates this region in the computation of the decision itself.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“decisions between reinforcers available at different costs result in elevated levels of activation in the executive system [48, 49].”
From The behavioral economics and neuroeconomics of reinforcer pathologies: implications for etiology and treatment of addiction (2011) · cites this paper as [49]
In the citing paper
“For example, lateral and ventromedial prefrontal cortices, which both encode the value of expected outcomes [67–68, 37] do show reduced activation with extended training [69–70].”
From Cortical and basal ganglia contributions to habit learning and automaticity (2010) · cites this paper as [68]
Citations
Cites
The assessment and analysis of handedness: The Edinburgh inventory
Oldfield, R. C. (1971) · Neuropsychologia · doi:10.1016/0028-3932(71)90067-4 · Cited by 35,900 · cited 1×
“edness inventory (62)] with no history of neurological or psychiatric illness par-”
BackgroundReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 2×
“Reinforcement learning theory stipulates that updating of action values occurs via a prediction error, which computes the difference between actual and expected outcomes (2, 39, 40).”
BackgroundReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Theory of games and economic behavior
(2019) · 100 Years of Math Milestones · doi:10.1090/mbk/121/32 · Cited by 16,632
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“Reinforcement learning theory stipulates that updating of action values occurs via a prediction error, which computes the difference between actual and expected outcomes (2, 39, 40).”
BackgroundTheory of Games and Economic Behavior
Hawkins, D. (1945) · Philosophy of Science · doi:10.1086/286866 · Cited by 3,887
Anterior Cingulate Cortex, Error Detection, and the Online Monitoring of Performance
Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., & Cohen, J. D. (1998) · Science · doi:10.1126/science.280.5364.747 · Cited by 3,273 · cited 1×
“An important question is the relationship between our suggested role for dmFC/ACC in the decision comparison process and prior findings implicating this area in error monitoring (35).”
ResultAnterior Cingulate Conflict Monitoring and Adjustments in Control
Kerns, J. G., Cohen, J. D., MacDonald, A. W., Cho, R. Y., Stenger, V. A., & Carter, C. S. (2004) · Science · doi:10.1126/science.1089910 · Cited by 2,891 · cited 1×
“Another pertinent issue is the extent to which the activity in dmFC/ACC is related to conflict monitoring, another cognitive function that has been attributed to this area (36).”
BackgroundUnconscious cerebral initiative and the role of conscious will in voluntary action
Libet, B. (1985) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00044903 · Cited by 2,623
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · cited 4×
“The model also assumed that action selection in every trial followed a soft-max probability rule based on the difference of the estimated action values (8).”
BackgroundResultInfluentialSelf-citation
Cited by
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
“…in dACC was found to closely track the extent to which the mean value of the alternative options was greater than that of the current options, and to correlate with the decision to switch option sets in such cases (see also Boorman et al., 2013; Rushworth et al., 2012;Wunderlich et al., 2009).”
Value, pleasure and choice in the ventral prefrontal cortex
Grabenhorst, F., & Rolls, E. T. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.12.004 · Cited by 766
Informatic parcellation of the network involved in the computation of subjective value
Clithero, J. A., & Rangel, A. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst106 · Cited by 706
Self-citationNeurophysiology of performance monitoring and adaptive behavior
Ullsperger, M., Danielmeier, C., & Jocham, G. (2014) · Physiological Reviews · doi:10.1152/physrev.00041.2012 · Cited by 634
Neural Basis of Reinforcement Learning and Decision Making
Lee, D., Seo, H., & Jung, M. W. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062111-150512 · Cited by 546
Neural computations associated with goal-directed choice
Rangel, A., & Hare, T. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.03.001 · Cited by 532
“Several human fMRI choice studies have found that BOLD activity in the OFC correlates with chosen values [46,69,70].”
BackgroundSelf-citationThe Role of Variability in Motor Learning
Dhawale, A. K., Smith, M. A., & Ölveczky, B. P. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031548 · Cited by 516
Cortical and basal ganglia contributions to habit learning and automaticity
Ashby, F. G., Turner, B. O., & Horvitz, J. C. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.02.001 · Cited by 481
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475
“Human fMRI studies report evidence that putative action-value signals are present in areas of the dorsal cortex, including the supplementary motor, lateral parietal, and dorsolateral cortices (Hare et al. 2011, Morris et al. 2014, Wunderlich et al. 2009).”
BackgroundSelf-citationDorsal Anterior Cingulate Cortex: A Bottom-Up View
Heilbronner, S. R., & Hayden, B. Y. (2016) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-070815-013952 · Cited by 455
“The dACC may serve as the site of comparison and selection in economic choice (Hare et al. 2011, Seo & Lee 2007, Wunderlich et al. 2009).”
Bias in the Brain: A Diffusion Model Analysis of Prior Probability and Potential Payoff
Mulder, M. J., Wagenmakers, E.-J., Ratcliff, R., Boekel, W., & Forstmann, B. U. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.4156-11.2012 · Cited by 421
“Other regions, such as the medFG and ACG, were sensitive to starting point changes as well.”
BackgroundResultInfluentialNeural representations of subjective reward value
Peters, J., & Büchel, C. (2010) · Behavioural Brain Research · doi:10.1016/j.bbr.2010.04.031 · Cited by 405
Transformation of stimulus value signals into motor commands during simple choice
Hare, T. A., Schultz, W., Camerer, C. F., O'Doherty, J. P., & Rangel, A. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1109322108 · Cited by 372
Self-citationSocial and monetary reward learning engage overlapping neural substrates
Lin, A., Adolphs, R., & Rangel, A. (2012) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsr006 · Cited by 351
“…in neuroscience studies of reinforcement learning (Pessiglione et al., 2006, 2008; Seymour et al., 2007; Lohrenz et al., 2007; Hare et al., 2008; Wunderlich et al., 2009) in which it is customary to fit the values of the SV signal based on the predictions of the best fitting learning model.”
MethodSelf-citationNeural mechanisms underlying motivation of mental versus physical effort
Schmidt, L., Lebreton, M., Cléry-Melin, M.-L., Daunizeau, J., & Pessiglione, M. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001266 · Cited by 310
“Recent reports have shown a distributed representation of effector-specific option values [38,57,58], suggesting that each task value would be encoded in a distinct prefrontal region.”
BackgroundMapping value based planning and extensively trained choice in the human brain
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3068 · Cited by 310
Self-citationDissociating valuation and saliency signals during decision-making
Litt, A., Plassmann, H., Shiv, B., & Rangel, A. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhq065 · Cited by 272
“It is important to emphasize that this potential confound is not a mere theoretical curiosity, since activity correlated with value has been found in areas traditionally associated with visual processing such as V1 (Serences 2008), areas involved in motor preparation such as the supplementary motor area (SMA) (Wunderl…”
BackgroundInfluentialSelf-citationAmping Up Effort: Effects of d-Amphetamine on Human Effort-Based Decision-Making
Wardle, M. C., Treadway, M. T., Mayo, L. M., Zald, D. H., & de Wit, H. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4387-11.2011 · Cited by 259
“While recent studies have made progress in elucidating brain regions underpinning these decisional processes in humans (Hare et al., 2008; Kable and Glimcher, 2009; Venkatraman et al., 2009; Wunderlich et al., 2009; Samanez-Larkin et al., 2010) concomitant neurochemical mechanisms remain poorly understood.”
BackgroundKlein-Flügge, M. C., Kennerley, S. W., Friston, K., & Bestmann, S. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.0292-16.2016 · Cited by 243
“Functionally dissociable anatomical subregions of mPFC The location in the dACC identified here is distinct from a more anterior and dorsal region in medial frontal cortex (in or near pre-SMA) where BOLD encodes the opposite signal: a negative value difference (Wunderlich et al., 2009; Hare et al., 2011).”
BackgroundResultInfluentialBickel, W. K., Jarmolowicz, D. P., Mueller, E. T., & Gatchalian, K. M. (2011) · Current Psychiatry Reports · doi:10.1007/s11920-011-0215-1 · Cited by 234
“Second, like decisions between immediate and delayed reinforcers, decisions between reinforcers available at different costs result in elevated levels of activation in the executive system [48, 49 ].”
BackgroundNeural systems underlying approach and avoidance in anxiety disorders
Robin L., A., & Martin, P. P. (2010) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2010.12.4/raupperle · Cited by 227
Contributions of the ventromedial prefrontal cortex to goal-directed action selection
O’Doherty, J. P. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06290.x · Cited by 225
Self-citationAn Agent Independent Axis for Executed and Modeled Choice in Medial Prefrontal Cortex
Nicolle, A., Klein-Flügge, M. C., Hunt, L. T., Vlaev, I., Dolan, R. J., & Behrens, T. E. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.07.023 · Cited by 224
Activity in dlPFC and its effective connectivity to vmPFC are associated with temporal discounting
Hare, T. A., Hakimi, S., & Rangel, A. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00050 · Cited by 206
Self-citationCamille, N., Tsuchida, A., & Fellows, L. K. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3164-11.2011 · Cited by 195
“There are few data in direct support of the claim that dACC is involved in action-value learning in humans (Gläscher et al., 2009; Wunderlich et al., 2009), although this region has been implicated in potentially related processes, including error monitoring, response inhibition, and self-generated action selection (G…”
BackgroundInfluentialA mechanism for value-guided choice based on the excitation-inhibition balance in prefrontal cortex
Jocham, G., Hunt, L. T., Near, J., & Behrens, T. E. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3140 · Cited by 183
Neural Correlates of Forward Planning in a Spatial Decision Task in Humans
Simon, D. A., & Daw, N. D. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4647-10.2011 · Cited by 177
“However, cortical correlates of chosen value are found in ventral prefrontal areas (Knutson et al., 2005; Hampton and O’Doherty, 2007; Kable and Glimcher, 2007; Chib et al., 2009; Gläscher et al., 2009; Wunderlich et al., 2009) more often than other areas (Breiter et al.”
BackgroundFouragnan, E., Retzler, C., & Philiastides, M. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24047 · Cited by 176
“Surprise Pattern B 41 (Allen et al., 2016; Amado et al., 2016; Amiez et al., 2012; Boll, Gamer, Gluth, Finsterbusch, & B€ uchel, 2013; Browning, Holmes, Murphy, Goodwin, & Harmer, 2010; Chumbley et al., 2014; Daw et al., 2011; Dreher, 2013; Ferdinand & Opitz, 2014; Forster & Brown, 2011; Fouragnan et al., 2015; Fourag…”
Decision Making: From Neuroscience to Psychiatry
Lee, D. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.008 · Cited by 166
“Consistent with the results from single-neuron recording studies (Sohn and Lee, 2007), signals related to values of reward expected from specific motor actions have been identified in the human supplementary motor area (Wunderlich et al., 2009).”
Boorman, E. D., Rushworth, M. F., & Behrens, T. E. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3022-12.2013 · Cited by 164
“Notably, however, as the difference between values of chosen and unchosen, or attended and unattended, options increases, the vmPFC signal increases whereas the DMFC signal decreases (Boorman et al., 2009; FitzGerald et al., 2009; Wunderlich et al., 2009; Lim et al., 2011).”
BackgroundChoosing Goals, Not Rules: Deciding among Rule-Based Action Plans
Klaes, C., Westendorff, S., Chakrabarti, S., & Gail, A. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.053 · Cited by 165
The functional neuroanatomy of decision making: Prefrontal control of thought and action
Coutlee, C. G., & Huettel, S. A. (2012) · Brain Research · doi:10.1016/j.brainres.2011.05.053 · Cited by 155
“Exploitative choices were associated with activation in the ventromedial PFC, consistent with the calculation of action value (Wunderlich et al., 2009).”
ResultNestor, L. J., Ghahremani, D. G., Monterosso, J., & London, E. D. (2011) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2011.04.010 · Cited by 151
“The SMC also plays an integral role in the planning of motor actions (Thaler et al., 1995; Lau et al., 2004; Campos et al., 2005; Seo et al., 2007; Wunderlich et al., 2009) and is closely related to the primary motor cortex (Romo et al.”
BackgroundDopamine and performance in a reinforcement learning task: evidence from Parkinson’s disease
Shiner, T., Seymour, B., Wunderlich, K., Hill, C., Bhatia, K. P., Dayan, P., & Dolan, R. J. (2012) · Brain · doi:10.1093/brain/aws083 · Cited by 149
“We were unable to dissociate whether the neural value correlates were precursors to choice (stimulus values) or the output of the choice process (chosen values) (Wunderlich et al., 2009).”
Self-citationValue normalization in decision making: theory and evidence
Rangel, A., & Clithero, J. A. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.07.011 · Cited by 143
“Other important signals include the net value of taking an action (action values [33]) and the values of chosen and unchosen options (for more details see [3–5]).”
BackgroundSelf-citationA proposed common neural mechanism for categorization and perceptual decisions
Freedman, D. J., & Assad, J. A. (2011) · Nature Neuroscience · doi:10.1038/nn.2740 · Cited by 162
Dissociation of memory systems: The story unfolds.
White, N. M., Packard, M. G., & McDonald, R. J. (2013) · Behavioral Neuroscience · doi:10.1037/a0034859 · Cited by 143
Neurons in Dorsal Anterior Cingulate Cortex Signal Postdecisional Variables in a Foraging Task
Blanchard, T. C., & Hayden, B. Y. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.3151-13.2014 · Cited by 141
“This information would presumably be used for a downstream comparison process, to determine a choice (Wunderlich et al., 2009).”
MethodEconomic choices can be made using only stimulus values
Wunderlich, K., Rangel, A., & O'Doherty, J. P. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1002258107 · Cited by 135
Self-citationIto, M., & Doya, K. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.04.001 · Cited by 160
“Action value In primates, the action value coding has been reported in DS [5,10,12 ,41,46], the internal pallidum [10], and the supplemental motor area [12 ].”
BackgroundO’Doherty, J. P. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.03.027 · Cited by 129
“Action-value signals have been reported in a number of brain regions including the striatum, lateral intra-parietal cortex and supplementary motor cortices (Lau and Glimcher, 2007; Platt and Glimcher, 1999; Samejima et al., 2005; Sohn and Lee, 2007; Sugrue et al., 2004; Wunderlich et al., 2009).”
BackgroundSelf-citationReputational Priors Magnify Striatal Responses to Violations of Trust
Fouragnan, E., Chierchia, G., Greiner, S., Neveu, R., Avesani, P., & Coricelli, G. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3086-12.2013 · Cited by 128
“As reported in previous studies, our results suggest that this brain network keeps track of contextually modulated decision values over trials, and doing so improves participants’ performance (Wunderlich et al., 2009).”
ResultNeural Basis of Motivational and Cognitive Control
(2011) · The MIT Press eBooks · doi:10.7551/mitpress/9780262016438.001.0001 · Cited by 128
Separate and overlapping brain areas encode subjective value during delay and effort discounting
Massar, S. A. A., Libedinsky, C., Weiyan, C., Huettel, S. A., & Chee, M. W. L. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.06.080 · Cited by 123
“Studies have shown that value computations during actionbased decisions are not necessarily supported by the vmPFC or ventral striatum, but rather by action specific brain areas (Wunderlich et al., 2009).”
Griffiths, K. R., Morris, R. W., & Balleine, B. W. (2014) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2014.00101 · Cited by 118
Decision-Making, Errors, and Confidence in the Brain
Rolls, E. T., Grabenhorst, F., & Deco, G. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00571.2010 · Cited by 116
Ito, M., & Doya, K. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.1962-14.2015 · Cited by 113
“Action value and state value coding Although neuronal correlation with action values has been reported mainly in DS (Samejima et al., 2005; Pasquereau et al., 2007; Lau and Glimcher, 2008; Hori et al., 2009; Kim et al., 2009; Wunderlich et al., 2009), clear differences in action-value representation among the subregio…”
BackgroundEconomic Choice as an Untangling of Options into Actions
Yoo, S. B. M., & Hayden, B. Y. (2018) · Neuron · doi:10.1016/j.neuron.2018.06.038 · Cited by 122
Models, movements, and minds: bridging the gap between decision making and action
Wispinski, N. J., Gallivan, J. P., & Chapman, C. S. (2018) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13973 · Cited by 119
A common currency for the computation of motivational values in the human striatum
Sescousse, G., Li, Y., & Dreher, J. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsu074 · Cited by 119
How prior preferences determine decision-making frames and biases in the human brain
Lopez‐Persem, A., Domenech, P., & Pessiglione, M. (2016) · eLife · doi:10.7554/elife.20317 · Cited by 119
“A spatial frame, based on the location of options (e.g., Vleft-Vright), has been suggested but not supported by much experimental evidence regarding the vmPFC valuation signal (Palminteri et al., 2009; Wunderlich et al., 2009; Skvortsova et al., 2014).”
BackgroundAnterior Prefrontal Cortex Contributes to Action Selection through Tracking of Recent Reward Trends
Kovach, C. K., Daw, N. D., Rudrauf, D., Tranel, D., O'Doherty, J. P., & Adolphs, R. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.5468-11.2012 · Cited by 100
“Relatively well established are the roles of ventromedial and orbital PFC in encoding reward value and dorsolateral PFC in linking rewards to cognitive control (Miller and Cohen, 2001), supported by functional imaging (O’Doherty et al., 2003; Wunderlich et al., 2009) and lesion data in humans (Bechara et al.”
BackgroundMethodSelf-citationThe cognitive neuroscience toolkit for the neuroeconomist: A functional overview.
Kable, J. W. (2011) · Journal of Neuroscience Psychology and Economics · doi:10.1037/a0023555 · Cited by 115
Nakao, T., Ohira, H., & Northoff, G. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00031 · Cited by 98
“…is modified based on the prediction error (i.e., discrepancies between expected and actual rewards; e.g., O’Doherty et al., 2004; Tanaka et al., 2004; Kim et al., 2006; Yoshida and Ishii, 2006; Behrens et al., 2007; Cohen, 2007; Boorman et al., 2009; Glascher et al., 2009; Wunderlich et al., 2009).”
BackgroundChoice Coding in Frontal Cortex during Stimulus-Guided or Action-Guided Decision-Making
Luk, C.-H., & Wallis, J. D. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4920-12.2013 · Cited by 96
“Furthermore, decisions in action space activate ACC (Wunderlich et al., 2009), whereas decisions in goods space activate ventromedial prefrontal cortex (Wunderlich et Figure 6.”
BackgroundDebates on the dorsomedial prefrontal/dorsal anterior cingulate cortex: insights for future research
Clairis, N., & Lopez‐Persem, A. (2023) · Brain · doi:10.1093/brain/awad263 · Cited by 122
“It roughly corresponds to the junction between Brodmann areas 4, 6, 24 and 32.”
BackgroundLutz, K., & Widmer, M. (2014) · Neuroscience and Neuroeconomics · doi:10.2147/nan.s38864 · Cited by 112
“This is partly consistent with studies showing brain activity in the ventromedial prefrontal cortex to correspond to the expected value of actions or choices.(34,35) These studies of reward-based learning make it clear that distinct mechanisms are likely to be involved when action is or is not required by the subject.”
ResultSupplementary Eye Field Encodes Option and Action Value for Saccades With Variable Reward
So, N.-Y., & Stuphorn, V. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00430.2010 · Cited by 93
“The relative location of pre-SMA, SMA, and SEF in this monkey confirms with earlier reports (Liu et al. 2002; Luppino et al. 1991, 1993; Matelli et al. 1991; Matsuzaka et al. 1992).”
BackgroundInfluentialThe Organization and Evolution of Dorsal Stream Multisensory Motor Pathways in Primates
Kaas, J. H., Gharbawie, O. A., & Stepniewska, I. (2011) · Frontiers in Neuroanatomy · doi:10.3389/fnana.2011.00034 · Cited by 90
“…motor areas, in conjunction with prefrontal cortex, contribute to the decision process (Romo et al., 2004; Schall et al., 2004; Schall, 2005; Wunderlich et al., 2009), and prefrontal cortex may be especially important when stored information rather than incoming sensory information…”
BackgroundBrennan, T. J., & Lo, A. W. (2011) · Quarterly Journal of Finance · doi:10.1142/s201013921100002x · Cited by 126
Computational models of adaptive behavior and prefrontal cortex
Soltani, A., & Koechlin, É. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01123-1 · Cited by 96
“Although action values (A-Rew) divorced from stimulus are experimentally dif fi cult to dissociate from stimulus values that elicit responses, human studies successfully dissociated the two value notions and found action values guiding selection to be encoded in the posterior dmPFC [101, 102].”
BackgroundWiers, C. E., Ludwig, V. U., Gladwin, T. E., Park, S. Q., Heinz, A., Wiers, R. W., Rinck, M., Lindenmeyer, J., Walter, H., & Bermpohl, F. (2015) · Addiction Biology · doi:10.1111/adb.12221 · Cited by 84
de Leeuw, M., Kahn, R. S., & Vink, M. (2015) · Schizophrenia Bulletin · doi:10.1093/schbul/sbu153 · Cited by 83
Zénon, A., Duclos, Y., Carron, R., Witjas, T., Baunez, C., Régis, J. M., Azulay, J., Brown, P., & Eusébio, A. (2016) · Brain · doi:10.1093/brain/aww075 · Cited by 98
“The number of 8-minute blocks of the control task varied between 2 and 4 in each treatment condition depending on time and fatigue constraints.”
The Interplay of Hippocampus and Ventromedial Prefrontal Cortex in Memory-Based Decision Making
Weilbächer, R. A., & Gluth, S. (2016) · Brain Sciences · doi:10.3390/brainsci7010004 · Cited by 86
Zaki, J., López, G., & Mitchell, J. P. (2013) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst005 · Cited by 82
“Studies of brain activity during the act of decision-making suggest that vMPFC activity tracks value associated with actions (e.g. the value of an outcome minus the costs of obtaining it; Kable and Glimcher, 2009; Wunderlich et al., 2009; Rangel and Hare, 2010).”
BackgroundBOLD Subjective Value Signals Exhibit Robust Range Adaptation
Cox, K. M., & Kable, J. W. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.3927-14.2014 · Cited by 75
“Although other fMRI studies have reported chosen value signals in the ACC and VMPFC (Gläscher et al., 2009; Wunderlich et al., 2009, 2010), our study was not optimized to examine adaptation using this alternative variable.”
MethodCross, L., Cockburn, J., Yue, Y., & O’Doherty, J. P. (2020) · Neuron · doi:10.1016/j.neuron.2020.11.021 · Cited by 87
“Our findings build on a growing catalog of intriguing similarities between DNNs and the brain (Eickenberg et al., 2017; G€ uçl€ u and vanGerven, 2015; Iigaya et al., 2020; Khaligh-Razavi and Kriegeskorte, 2014; Wang et al., 2018; Wen et al., 2018; Yamins et al., 2014; Yamins and DiCarlo, 2016).”
Self-citationCorrelates of decisional dynamics in the dorsal anterior cingulate cortex
Azab, H., & Hayden, B. Y. (2017) · PLoS Biology · doi:10.1371/journal.pbio.2003091 · Cited by 80
Correlates of economic decisions in the dorsal and subgenual anterior cingulate cortices
Azab, H., & Hayden, B. Y. (2018) · European Journal of Neuroscience · doi:10.1111/ejn.13865 · Cited by 77
“Others view the dACC as a crucial part of the choice process (Bush et al., 2002; Walton et al., 2006; Croxson et al., 2009; Wunderlich et al., 2009; Hillman & Bilkey, 2010; Hare et al., 2011; Boorman et al., 2013; Grueschow et al., 2015; Klein-Flugge et al., 2016; Hayden et al., 2011; Azab &…”
BackgroundAction-specific value signals in reward-related regions of the human brain
FitzGerald, T. H. B., Friston, K. J., & Dolan, R. J. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.3254-12.2012 · Cited by 68
“In humans, there is evidence of such representations in motor areas (Wunderlich et al., 2009), but not those most often associated with valuation, such as the striatum.”
BackgroundMethodResultInfluentialModel-based action planning involves cortico-cerebellar and basal ganglia networks
Fermin, A. S. R., Yoshida, T., Yoshimoto, J., Ito, M., Tanaka, S. C., & Doya, K. (2016) · Scientific Reports · doi:10.1038/srep31378 · Cited by 67
Stimulus devaluation induced by stopping action.
Wessel, J. R., O’Doherty, J. P., Berkebile-Weinberg, M., Linderman, D., & Aron, A. R. (2014) · Journal of Experimental Psychology General · doi:10.1037/xge0000022 · Cited by 71
Self-citationShahar, N., Moran, R., Hauser, T. U., Kievit, R. A., McNamee, D., Moutoussis, M., NSPN Consortium, Dolan, R. J., Bullmore, E., Dolan, R., et al. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821647116 · Cited by 65
“This conclusion is in line with diverse findings from animal studies (21 – 23), focal brain damage (24, 25), and human learning studies (26, 27), which seem to show that stimulus and action-related value encoding are realized within distinct neural networks.”
ResultNeural computations underlying inverse reinforcement learning in the human brain
Collette, S., Pauli, W. M., Bossaerts, P. L., & O’Doherty, J. P. (2017) · eLife · doi:10.7554/elife.29718 · Cited by 65
Self-citationThe Neural Representation of Voluntary Task-Set Selection in Dynamic Environments
Wisniewski, D., Reverberi, C., Tusche, A., & Haynes, J.-D. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu155 · Cited by 62
“Furthermore, it has been shown that the dmPFC compares the values of different choice options (Wunderlich et al. 2009; Hare et al. 2011).”
BackgroundFouragnan, E., Queirazza, F., Retzler, C., Mullinger, K. J., & Philiastides, M. G. (2017) · Scientific Reports · doi:10.1038/s41598-017-04507-w · Cited by 61
“Small panel: A Bayesian model comparison using BIC scores revealed that the Model-free (Mf) RL model explained the data better than a Model-based (Mb) RL model and a model including a dynamic learning rate (Dy) (Lower BIC scores indicate a better fit – see Methods).”
InfluentialDubljević, V., & Racine, É. (2014) · AJOB Neuroscience · doi:10.1080/21507740.2014.939381 · Cited by 69
“…and nonhuman animal decision making there is a vast neuroscience literature on “actionvalue” and the importance of inhibiting certain responses (e.g., Wunderlich, Rangel, and O’Doherty 2009), which might illuminate the neural mechanisms underpinning deontological insights and intuitions about…”
BackgroundChoi, Y., Shin, E. Y., & Kim, S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2003963117 · Cited by 65
Morris, R. W., Dezfouli, A., Griffiths, K. R., & Balleine, B. W. (2014) · Nature Communications · doi:10.1038/ncomms5390 · Cited by 59
Stott, J. J., & Redish, A. D. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0472 · Cited by 73
Ballard, I. C., & McClure, S. M. (2019) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2019.01.006 · Cited by 68
Li, V., Michael, E., Balaguer, J., Castañón, S. H., & Summerfield, C. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1805224115 · Cited by 59
Prefrontal mechanisms combining rewards and beliefs in human decision-making
Rouault, M., Drugowitsch, J., & Koechlin, É. (2019) · Nature Communications · doi:10.1038/s41467-018-08121-w · Cited by 67
Bellucci, G., Hahn, T., Deshpande, G., & Krueger, F. (2019) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-018-00654-3 · Cited by 55
Striatal action-value neurons reconsidered
Elber-Dorozko, L., & Loewenstein, Y. (2018) · eLife · doi:10.7554/elife.34248 · Cited by 53
Jocham, G., Furlong, P. M., Kröger, I. L., Kahn, M. C., Hunt, L. T., & Behrens, T. E. J. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.06.005 · Cited by 53
“Whilst neural signals representing the potential value of different courses of action are widespread throughout the brain (Cai et al., 2011; Dorris and Glimcher, 2004; Hernandez et al., 2002; Kable and Glimcher, 2007; Kim et al., 2008; Padoa-Schioppa and Assad, 2006; Platt and Glimcher, 1999; Serences, 2008; Sugrue et…”
BackgroundFunctional and Structural Brain Changes Associated with Methamphetamine Abuse
Jan, R. K., Kydd, R. R., & Russell, B. R. (2012) · Brain Sciences · doi:10.3390/brainsci2040434 · Cited by 59
“Hypoactivation of the SMA of MA abusers during the incongruent condition was thought to be related to impaired planning of motor actions [172], as well as response to conflict [173] and action-based decision-making [174].”
BackgroundFrom Feedback- to Response-based Performance Monitoring in Active and Observational Learning
Bellebaum, C., & Colosio, M. (2014) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00612 · Cited by 59
Valuation, Intertemporal Choice, and Self-Control
Kable, J. W. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00010-3 · Cited by 66
“…and argued that such a signal is consistent 2 0 P op ul at io n ac tiv ity (n or m ) 0 500 1000 0 500 1000 2 0 Time (ms) 2 0 2 0 v s v sMonkey W Monkey D NEUROECONOMICS with a region that encodes action value comparisons at the single neuron level (Hare et al., 2011b; Wunderlich et al., 2009).”
BackgroundUsing Data-Driven Model-Brain Mappings to Constrain Formal Models of Cognition
Borst, J. P., Nijboer, M., Taatgen, N., van Rijn, H., & Anderson, J. R. (2015) · PLoS ONE · doi:10.1371/journal.pone.0119673 · Cited by 55
“Such models have for instance been used to locate regions involved in reinforcement learning [29–32], category learning [33], and decision making [34,35].”
MethodBridging Levels of Understanding in Schizophrenia Through Computational Modeling
Anticevic, A., Murray, J. D., & Barch, D. M. (2015) · Clinical Psychological Science · doi:10.1177/2167702614562041 · Cited by 53
Brain Systems for Probabilistic and Dynamic Prediction: Computational Specificity and Integration
O'Reilly, J. X., Jbabdi, S., Rushworth, M. F. S., & Behrens, T. E. J. (2013) · PLoS Biology · doi:10.1371/journal.pbio.1001662 · Cited by 48
“This finding is analogous to previous observations [51,52] that motoric regions of cortex show reward prediction error signals in tasks in which the level of reward is determined by the choice of action.”
ResultValue-free reinforcement learning: policy optimization as a minimal model of operant behavior
Bennett, D., Niv, Y., & Langdon, A. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.04.020 · Cited by 55
Cooper, J. C., Dunne, S., Furey, T., & O'Doherty, J. P. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.2558-12.2012 · Cited by 45
“…in decision making for simple choices, where its activity is related to subsequent decisions or comparisons between options (Hampton et al., 2008; Wunderlich et al., 2009; Hare et al., 2011); it is also implicated in controlling social behavior in nonhuman primates (Hadland et al., 2003;…”
BackgroundSelf-citationDecision-making under risk: A graph-based network analysis using functional MRI
Minati, L., Grisoli, M., Seth, A. K., & Critchley, H. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.048 · Cited by 48
“therefore strongly endorse the view that the mesial prefrontal cortex acts as fundamentally a decisional comparator (Venkatraman et al., 2009; Wunderlich et al., 2009).”
BackgroundResultInfluentialPanasiti, M. S., Pavone, E. F., Mancini, A., Merla, A., Grisoni, L., & Aglioti, S. M. (2014) · Social Neuroscience · doi:10.1080/17470919.2014.934394 · Cited by 66
Advanced Reinforcement Learning
Daw, N. D. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00016-4 · Cited by 54
“Accordingly, in an fMRI study, distinct prediction errors were seen in left and right striatum for movements of the contralateral hand, in a task where these had separable values (Gershman et al., 2009; see also Palminteri et al., 2009; Wunderlich et al., 2009).”
BackgroundNeural basis of learning and preference during social decision-making
Seo, H., & Lee, D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.010 · Cited by 42
Brain mechanisms of social comparison and their influence on the reward system
Kedia, G., Mussweiler, T., & Linden, D. E. J. (2014) · Neuroreport · doi:10.1097/wnr.0000000000000255 · Cited by 52
“This network, and in particular the IPS, is recruited by comparisons of numbers presented in different formats (e.g., digits, words), size, dot patterns, line lengths, magnitudes of angles, luminance, and money rewards [60 – 67].”
Background
Metadata
- Authors
- Wunderlich, K., Rangel, A., & O'Doherty, J. P.
- Affiliations
- California Institute of Technology (US)
- Computation and Neural Systems Program, California Institute of Technology, Pasadena, CA;
- Corresponding author
- Wunderlich, Klaus
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2009-10-06
- Volume
- 106
- Issue
- 40
- Pages
- 17199-17204
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC2761331
- OpenAlex
- W2117402031
- Cited by
- 301
- References
- 106
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 39 · 2023: 12 · 2022: 10 · 2021: 17 · 2020: 14 · 2019: 10 · 2018: 15 · 2017: 12 · 2016: 19 · 2015: 15 · 2014: 31 · 2013: 22 · 2012: 25
- FWCI
- 10.65
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- action value, supplementary motor area, ventromedial prefrontal cortex, dorsomedial prefrontal cortex, expected value, reward
- MeSH terms
- Adolescent, Adult, Brain, Brain Mapping, Choice Behavior, Decision Making, Eye Movements, Female, Humans, Magnetic Resonance Imaging, Male, Models, Neurological, Photic Stimulation, Psychomotor Performance, Reward, Task Performance and Analysis, Prefrontal Cortex, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.