Expectancy, ambiguity, and behavioral flexibility: separable and complementary roles of the orbital frontal cortex and amygdala in processing reward expectancies
Abstract
Appetitive goal-directed behavior can be associated with a cue-triggered expectancy that it will lead to a particular reward, a process thought to depend on the OFC and basolateral amygdala complex. We developed a biologically informed neural network model of this system to investigate the separable and complementary roles of these areas as the main components of a flexible expectancy system. These areas of interest are part of a neural network with additional subcortical areas, including the central nucleus of amygdala, ventral (limbic) and dorsomedial (associative) striatum. Our simulations are consistent with the view that the amygdala maintains Pavlovian associations through incremental updating of synaptic strength and that the OFC supports flexibility by maintaining an activation-based working memory of the recent reward history. Our model provides a mechanistic explanation for electrophysiological evidence that cue-related firing in OFC neurons is nonselectively early after a contingency change and why this nonselective firing is critical for promoting plasticity in the amygdala. This ambiguous activation results from the simultaneous maintenance of recent outcomes and obsolete Pavlovian contingencies in working memory. Furthermore, at the beginning of reversal, the OFC is critical for supporting responses that are no longer inappropriate. This result is inconsistent with an exclusive inhibitory account of OFC function.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The OFC is involved in the processing of reward-related information.
“The OFC was also found to be involved in the processing of reward-related information (Schoenbaum and Roesch, 2005; Pauli et al., 2012).”
From Functional neuroimaging of visuospatial working memory tasks enables accurate detection of attention deficit and hyperactivity disorder (2015) · cites this paper as Pauli et al., 2012
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
(1994) · DigitalGeorgetown (Georgetown University Library) · Cited by 10,542 · see citing works
Parallel organization of functionally segregated circuits linking basal ganglia and cortex
Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.09.030186.002041 · Cited by 8,527 · see citing works
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 7,268 · see citing works
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855 · see citing works
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · see citing works
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785 · see citing works
Ongur, D. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.206 · Cited by 2,627 · see citing works
THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS
Mink, J. W. (1996) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00042-1 · Cited by 2,478 · see citing works
Simplified neuron model as a principal component analyzer
Oja, E. (1982) · Journal of Mathematical Biology · doi:10.1007/bf00275687 · Cited by 2,309 · see citing works
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Cardinal, R. N., Parkinson, J. A., Hall, J., & Everitt, B. J. (2002) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(02)00007-6 · Cited by 2,129 · see citing works
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964 · see citing works
Self-citationThe principal features and mechanisms of dopamine modulation in the prefrontal cortex
Seamans, J. K., & Yang, C. R. (2004) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2004.05.006 · Cited by 1,592 · see citing works
Dissociation in prefrontal cortex of affective and attentional shifts
Dias, R., Robbins, T. W., & Roberts, A. C. (1996) · Nature · doi:10.1038/380069a0 · Cited by 1,527 · see citing works
Descartes' Error: Emotion, Reason, and the Human Brain
Joynt, R. J. (1995) · JAMA · doi:10.1001/jama.1995.03520420079045 · Cited by 1,447 · see citing works
The Role of the Dorsal Striatum in Reward and Decision-Making: Figure 1.
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1554-07.2007 · Cited by 1,390 · see citing works
The organization of the projection from the cerebral cortex to the striatum in the rat
McGeorge, A. J., & Faull, R. L. M. (1989) · Neuroscience · doi:10.1016/0306-4522(89)90128-0 · Cited by 1,345 · see citing works
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291 · see citing works
The amygdala: Neurobiological aspects of emotion, memory and mental dysfunction
Scheel-Krüger, J. (1993) · Journal of Chemical Neuroanatomy · doi:10.1016/0891-0618(93)90037-5 · Cited by 1,277 · see citing works
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190 · see citing works
Dichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190 · see citing works
Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex
Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2003) · Science · doi:10.1126/science.1087919 · Cited by 1,165 · see citing works
Adolphs, R., Tranel, D., Damasio, H., & Damasio, A. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-09-05879.1995 · Cited by 1,138 · see citing works
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · see citing works
Floresco, S. B., West, A. R., Ash, B., Moore, H., & Grace, A. A. (2003) · Nature Neuroscience · doi:10.1038/nn1103 · Cited by 1,106 · see citing works
The role of the dorsomedial striatum in instrumental conditioning
Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04218.x · Cited by 1,099 · see citing works
Cortical mechanisms of action selection: the affordance competition hypothesis
Cisek, P. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2054 · Cited by 1,077 · see citing works
O’Reilly, R. C., & Frank, M. J. (2005) · Neural Computation · doi:10.1162/089976606775093909 · Cited by 1,066 · see citing works
Self-citationArchitectonic subdivision of the human orbital and medial prefrontal cortex
Öngür, D., Ferry, A. T., & Price, J. L. (2003) · Journal of Comparative Neurology · doi:10.1002/cne.10609 · Cited by 944 · see citing works
Orbitofrontal cortex and its contribution to decision-making.
Wallis, J. D. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094334 · Cited by 938 · see citing works
Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens
Nicola, S. M., Surmeier, D. J., & Malenka, R. C. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.185 · Cited by 931 · see citing works
Frank, M. J. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929052880093 · Cited by 925 · see citing works
The primate amygdala represents the positive and negative value of visual stimuli during learning
Paton, J. J., Belova, M. A., Morrison, S. E., & Salzman, C. D. (2006) · Nature · doi:10.1038/nature04490 · Cited by 917 · see citing works
Why We Think Plasticity Underlying Pavlovian Fear Conditioning Occurs in the Basolateral Amygdala
Fanselow, M. S., & LeDoux, J. E. (1999) · Neuron · doi:10.1016/s0896-6273(00)80775-8 · Cited by 914 · see citing works
Elliott, R. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.308 · Cited by 887 · see citing works
Interactions between frontal cortex and basal ganglia in working memory: A computational model
FRANK, M. J., LOUGHRY, B., & O'REILLY, R. C. (2001) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.1.2.137 · Cited by 879 · see citing works
Self-citationSynaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory
Wang, X.-J. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-21-09587.1999 · Cited by 859 · see citing works
Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning
Schoenbaum, G. M., Chiba, A. A., & Gallagher, M. (1998) · Nature Neuroscience · doi:10.1038/407 · Cited by 849 · see citing works
(1964) · McGraw-Hill eBooks · Cited by 838 · see citing works
The orbitofrontal cortex: Neuronal activity in the behaving monkey
Thorpe, S. J., Rolls, E. T., & Maddison, S. (1983) · Experimental Brain Research · doi:10.1007/bf00235545 · Cited by 836 · see citing works
The orbitofrontal cortex and beyond: From affect to decision-making
Rolls, E. T., & Fabian Grabenhorst, B. (2008) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2008.09.001 · Cited by 808 · see citing works
Quirk, G. J., Repa, J. C., & LeDoux, J. E. (1995) · Neuron · doi:10.1016/0896-6273(95)90092-6 · Cited by 808 · see citing works
Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792 · see citing works
Ghashghaei, H. T., & Barbas, H. (2002) · Neuroscience · doi:10.1016/s0306-4522(02)00446-3 · Cited by 785 · see citing works
Rolls, E. T. (1996) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.1996.0128 · Cited by 755 · see citing works
Neurocomputational models of working memory
Durstewitz, D., Seamans, J. K., & B, T. J. S. A. (2000) · Nature Neuroscience · doi:10.1038/81460 · Cited by 710 · see citing works
The frontal granular cortex and behavior
Pribram, K. H. (1965) · Neuropsychologia · doi:10.1016/0028-3932(65)90022-9 · Cited by 705 · see citing works
The amygdala, reward and emotion
Murray, E. A. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.08.013 · Cited by 679 · see citing works
Do Rats Have Prefrontal Cortex? The Rose-Woolsey-Akert Program Reconsidered
Preuss, T. M. (1995) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1995.7.1.1 · Cited by 666 · see citing works
Basal Ganglia Output and Cognition: Evidence from Anatomical, Behavioral, and Clinical Studies
Middleton, F. A., & Strick, P. L. (2000) · Brain and Cognition · doi:10.1006/brcg.1999.1099 · Cited by 657 · see citing works
A computational model of action selection in the basal ganglia. I. A new functional anatomy
GURNEY, K. N., Prescott, T. J., & Redgrave, P. (2001) · Biological Cybernetics · doi:10.1007/pl00007984 · Cited by 633 · see citing works
(1992) · Cited by 626 · see citing works
Joel, D., & Weiner, I. (2000) · Neuroscience · doi:10.1016/s0306-4522(99)00575-8 · Cited by 614 · see citing works
The Cognitive Neuroscience of Emotion
Watt, D. F. (2001) · Neuropsychoanalysis · doi:10.1080/15294145.2001.10773346 · Cited by 610 · see citing works
Houk, J. C., & Wise, S. P. (1995) · Cerebral Cortex · doi:10.1093/cercor/5.2.95 · Cited by 594 · see citing works
A new perspective on the role of the orbitofrontal cortex in adaptive behaviour
Schoenbaum, G., Roesch, M. R., Stalnaker, T. A., & Takahashi, Y. K. (2009) · Nature reviews. Neuroscience · doi:10.1038/nrn2753 · Cited by 585 · see citing works
Schoenbaum, G., Chiba, A. A., & Gallagher, M. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-05-01876.1999 · Cited by 579 · see citing works
Postuma, R. B., & Dagher, A. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj088 · Cited by 579 · see citing works
Frank, M. J., & Claus, E. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.300 · Cited by 578 · see citing works
Computational Explorations in Cognitive Neuroscience
O’Reilly, R. C., & Munakata, Y. (2000) · The MIT Press eBooks · doi:10.7551/mitpress/2014.001.0001 · Cited by 564 · see citing works
Self-citationA Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement
Houk, J. C., Adams, J. L., & Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0020 · Cited by 555 · see citing works
Hatfield, T., Han, J.-S., Conley, M., Gallagher, M., & Holland, P. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-16-05256.1996 · Cited by 555 · see citing works
Semba, K., & Fibiger, H. C. (1992) · The Journal of Comparative Neurology · doi:10.1002/cne.903230307 · Cited by 544 · see citing works
Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour
Rushworth, M. F. S., Behrens, T. E. J., Rudebeck, P. H., & Walton, M. E. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.01.004 · Cited by 507 · see citing works
Hernández‐López, S., Bargas, J., Surmeier, D. J., Reyes, A., & Galarraga, E. (1997) · Journal of Neuroscience · doi:10.1523/jneurosci.17-09-03334.1997 · Cited by 495 · see citing works
Schoenbaum, G., Setlow, B., Saddoris, M. P., & Gallagher, M. (2003) · Neuron · doi:10.1016/s0896-6273(03)00474-4 · Cited by 484 · see citing works
Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2055 · Cited by 450 · see citing works
Self-citationBiologically Based Computational Models of High-Level Cognition
O’Reilly, R. C. (2006) · Science · doi:10.1126/science.1127242 · Cited by 447 · see citing works
Self-citationA role for NMDA-receptor channels in working memory
Lisman, J. E., Fellous, J., & Wang, X. (1998) · Nature Neuroscience · doi:10.1038/1086 · Cited by 438 · see citing works
How does PKMζ maintain long-term memory?
Sacktor, T. C. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn2949 · Cited by 431 · see citing works
Orbitofrontal cortex, associative learning and expectancies
Schoenbaum, G., & Roesch, M. (2005) · Neuron · doi:10.1016/j.neuron.2005.07.018 · Cited by 404 · see citing works
Stop-Signal Reaction-Time Task Performance: Role of Prefrontal Cortex and Subthalamic Nucleus
Eagle, D. M., Baunez, C., Hutcheson, D. M., Lehmann, O., Shah, A. P., & Robbins, T. W. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm044 · Cited by 404 · see citing works
Amygdala–frontal interactions and reward expectancy
Holland, P. C., & Gallagher, M. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.03.007 · Cited by 390 · see citing works
Banishing the homunculus: Making working memory work
Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2006) · Neuroscience · doi:10.1016/j.neuroscience.2005.04.067 · Cited by 385 · see citing works
Self-citationOrbitofrontal Cortex Mediates Outcome Encoding in Pavlovian But Not Instrumental Conditioning
Ostlund, S. B., & Balleine, B. W. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5443-06.2007 · Cited by 379 · cited 1× · see citing works
“More generally, the expectancy system is critical for modulating behavior as a function of changing needs and goals, as explored, for example, in devaluation and related paradigms (Ostlund & Balleine, 2007 ).”
BackgroundA Physiologically Plausible Model of Action Selection and Oscillatory Activity in the Basal Ganglia
Humphries, M. D., Stewart, R. D., & GURNEY, K. N. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.3486-06.2006 · Cited by 364 · see citing works
O'Reilly, R. C. (1996) · Neural Computation · doi:10.1162/neco.1996.8.5.895 · Cited by 352 · see citing works
Self-citationMolecular effects of dopamine on striatal-projection pathways
Gerfen, C. R. (2000) · Trends in Neurosciences · doi:10.1016/s1471-1931(00)00019-7 · Cited by 341 · see citing works
Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J., & Rushworth, M. F. S. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012246107 · Cited by 340 · see citing works
The amygdala, fear, and memory
FANSELOW, M. S., & GALE, G. D. (2003) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2003.tb07077.x · Cited by 299 · see citing works
Lewis, B. L. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.12.1168 · Cited by 287 · see citing works
Takahashi, Y. K., Roesch, M. R., Stalnaker, T. A., Haney, R. Z., Calu, D. J., Taylor, A. R., Burke, K. A., & Schoenbaum, G. M. (2009) · Neuron · doi:10.1016/j.neuron.2009.03.005 · Cited by 284 · see citing works
Models of information processing in the basal ganglia
Alexander, G. E. (1996) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0921-884x(96)89703-7 · Cited by 278 · see citing works
Clarke, H. F., Robbins, T. W., & Roberts, A. C. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1521-08.2008 · Cited by 263 · see citing works
Brown, J. W., Bullock, D. H., & Grossberg, S. (2003) · Neural Networks · doi:10.1016/j.neunet.2003.08.006 · Cited by 263 · see citing works
Morton, J. B., & Munakata, Y. (2002) · Developmental Psychobiology · doi:10.1002/dev.10033 · Cited by 260 · see citing works
Surround inhibition among projection neurons is weak or nonexistent in the rat neostriatum
Jaeger, D., Kita, H., & Wilson, C. J. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.5.2555 · Cited by 232 · see citing works
A Computational Model of How the Basal Ganglia Produce Sequences
Berns, G. S., & B, T. J. S. A. (1998) · Journal of Cognitive Neuroscience · doi:10.1162/089892998563815 · Cited by 231 · see citing works
Rapid Associative Encoding in Basolateral Amygdala Depends on Connections with Orbitofrontal Cortex
Saddoris, M. P., Gallagher, M., & Schoenbaum, G. M. (2005) · Neuron · doi:10.1016/j.neuron.2005.02.018 · Cited by 214 · see citing works
Cognitive-Emotional Interactions: Listen to the Brain
LeDoux, J. E. (1999) · doi:10.1093/oso/9780195118889.003.0007 · Cited by 210 · see citing works
Action selection and refinement in subcortical loops through basal ganglia and cerebellum
Houk, J. C., Bastianen, C., Fansler, D., Fishbach, A., Fraser, D. A., Reber, P. J., Roy, S. A., & Simó, L. S. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2063 · Cited by 196 · see citing works
A Model of Corticostriatal Plasticity for Learning Oculomotor Associations and Sequences
Dominey, P. F., Arbib, M. A., & Joseph, J. (1995) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1995.7.3.311 · Cited by 191 · see citing works
Corbit, L. H., Muir, J. L., & Balleine, B. W. (2003) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2003.02833.x · Cited by 186 · see citing works
Basolateral Amygdala Lesions Abolish Orbitofrontal-Dependent Reversal Impairments
Stalnaker, T. A., Franz, T. M., Singh, T., & Schoenbaum, G. M. (2007) · Neuron · doi:10.1016/j.neuron.2007.02.014 · Cited by 178 · see citing works
Gorelova, N. A., & Yang, C. R. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.1.75 · Cited by 162 · see citing works
(1993) · Elsevier eBooks · doi:10.1016/c2009-0-35344-1 · Cited by 161 · see citing works
PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm.
O’Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007) · Behavioral Neuroscience · doi:10.1037/0735-7044.121.1.31 · Cited by 144 · see citing works
Self-citationThe central nucleus of the amygdala projection to dopamine subpopulations in primates
Fudge, J. L., & Haber, S. N. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00092-0 · Cited by 138 · see citing works
Representation of olfactory information in the primate orbitofrontal cortex
Rolls, E. T., Critchley, H., & Treves, A. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.75.5.1982 · Cited by 136 · see citing works
Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning
O’Reilly, R. C. (2001) · Neural Computation · doi:10.1162/08997660152002834 · Cited by 133 · see citing works
Self-citationWallis, J. D., Dias, R. S., Robbins, T. W., & Roberts, A. (2001) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01546.x · Cited by 113 · see citing works
Neural mechanisms of acquired phasic dopamine responses in learning
Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2009) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.11.019 · Cited by 103 · see citing works
Self-citationFeatherstone, R. E., & McDonald, R. J. (2003) · Behavioural Brain Research · doi:10.1016/s0166-4328(03)00218-3 · Cited by 98 · see citing works
Stalnaker, T. A., Roesch, M. R., Franz, T. M., Burke, K. A., & Schoenbaum, G. M. (2006) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2006.05128.x · Cited by 83 · see citing works
Gray, T. S. (1999) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1999.tb09281.x · Cited by 69 · see citing works
Ahn, S., & Phillips, A. G. (2003) · Neuroscience · doi:10.1016/s0306-4522(02)00551-1 · Cited by 68 · see citing works
Toshiaki, K. I. T. A., Nishijo, H., Eifuku, S., TERASAWA, K., & Ono, T. (1995) · Journal of Neuroscience · doi:10.1523/jneurosci.15-03-01683.1995 · Cited by 64 · see citing works
Effects of local connectivity on striatal function: Simulation and analysis of a model
Wickens, J. R., Kötter, R., & Alexander, M. E. (1995) · Synapse · doi:10.1002/syn.890200402 · Cited by 64 · see citing works
Chudasama, Y., Kralik, J. D., & Murray, E. A. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl025 · Cited by 59 · see citing works
The contribution of NMDA receptors in the dorsolateral striatum to egocentric response learning.
Palencia, C. A., & Ragozzino, M. E. (2005) · Behavioral Neuroscience · doi:10.1037/0735-7044.119.4.953 · Cited by 49 · see citing works
Costumero, V., Barrós‐Loscertales, A., Bustamante, J. C., Ventura‐Campos, N., Fuentes‐Claramonte, P., & Ávila, C. (2013) · European Journal of Neuroscience · doi:10.1111/ejn.12234 · Cited by 44 · see citing works
Rouillard, C., & Freeman, A. S. (1995) · Synapse · doi:10.1002/syn.890210410 · Cited by 16 · see citing works
Dilmore, J. G., Gutkin, B. S., & Ermentrout, G. B. (1999) · Neurocomputing · doi:10.1016/s0925-2312(99)00005-3 · Cited by 13 · see citing works
Flagged first, then by citations
2 of the 117 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Whitney, P. J., Hinson, J. M., Jackson, M. L., & Dongen, H. P. A. V. (2015) · SLEEP · doi:10.5665/sleep.4668 · Cited by 145 · see citing works
Orbitofrontal cortex volume and brain reward response in obesity
Shott, M. E., Cornier, M.-A., Mittal, V. A., Pryor, T. L., Orr, J. M., Brown, M. S., & Frank, G. (2014) · International Journal of Obesity · doi:10.1038/ijo.2014.121 · Cited by 129 · see citing works
Unraveling the Mysteries of Motivation
O’Reilly, R. C. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.03.001 · Cited by 48 · see citing works
Self-citationReward Sensitivity Is Associated with Brain Activity during Erotic Stimulus Processing
Costumero, V., Barrós-Loscertales, A., Bustamante, J. C., Ventura-Campos, N., Fuentes, P., Rosell-Negre, P., & Ávila, C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066940 · Cited by 45 · see citing works
Costumero, V., Barrós‐Loscertales, A., Bustamante, J. C., Ventura‐Campos, N., Fuentes‐Claramonte, P., & Ávila, C. (2013) · European Journal of Neuroscience · doi:10.1111/ejn.12234 · Cited by 44 · see citing works
Pauli, W. M., Clark, A. D., Guenther, H. J., O’Reilly, R. C., & Rudy, J. W. (2012) · Learning & Memory · doi:10.1101/lm.025148.111 · Cited by 44 · see citing works
Self-citationThe Leabra Cognitive Architecture: How to Play 20 Principles with Nature and Win!
O’Reilly, R. C., Hazy, T. E., & Herd, S. A. (2012) · Cited by 39 · see citing works
Self-citationA pavlovian model of the amygdala and its influence within the medial temporal lobe
Carrere, M., & Alexandre, F. (2015) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2015.00041 · Cited by 36 · see citing works
Multilevel models from biology to psychology: Mission impossible?
Bilder, R. M., Howe, A. G., & Sabb, F. W. (2013) · Journal of Abnormal Psychology · doi:10.1037/a0032263 · Cited by 36 · see citing works
Sleep loss, executive function, and decision-making
Satterfield, B. C., & Killgore, W. D. S. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-815373-4.00026-5 · Cited by 35 · see citing works
Goal-Driven Cognition in the Brain: A Computational FrameworkPreprint
O’Reilly, R. C., Hazy, T. E., Mollick, J., Mackie, P., & Herd, S. A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.7591 · Cited by 45 · see citing works
Self-citationRodrigues, B., Coelho, A., Portugal‐Nunes, C., Magalhães, R. J. D. S., Moreira, P. S., Castanho, T. C., Amorim, L., Marques, P., Soares, J. M., Sousa, N., et al. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00786 · Cited by 32 · see citing works
Hammer, R., Cooke, G. E., Stein, M. A., & Booth, J. R. (2015) · NeuroImage: Clinical · doi:10.1016/j.nicl.2015.08.015 · Cited by 25 · see citing works
A Systems-Neuroscience Model of Phasic Dopamine
Mollick, J., Hazy, T., Krueger, K. A., Nair, A., Mackie, P., Herd, S. A., & O'Reilly, R. (2020) · Psychology Review · doi:10.1037/rev0000199 · Cited by 20 · see citing works
Self-citationHammer, R., Tennekoon, M., Cooke, G. E., Gayda, J., Stein, M. A., & Booth, J. R. (2015) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2015.06.002 · Cited by 21 · see citing works
Huang, Y., Mao, M., Zong, Z., Zhou, H., Zhao, Y., Duan, L. F., Kreplin, U., & Xiao, X. N. (2017) · Journal of Biomedical Optics · doi:10.1117/1.jbo.22.1.016011 · Cited by 22 · see citing works
Understanding Neuronal Architecture in Obesity through Analysis of White Matter Connection Strength
Riederer, J., Shott, M. E., DeGuzman, M. C., Pryor, T. L., & Frank, G. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00271 · Cited by 20 · see citing works
Computational approaches to habits in a model-free world
Pauli, W. M., Cockburn, J., Pool, E. R., Perez, O. D., & O’Doherty, J. P. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.12.001 · Cited by 16 · see citing works
Self-citationOn the Complexity of Brain Disorders: A Symptom-Based Approach
Moustafa, A. A., Phillips, J. R., Kéri, S., Misiak, B., & Frydecka, D. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00016 · Cited by 14 · see citing works
Stippekohl, B., Winkler, M. H., Walter, B., Kagerer, S., Mucha, R. F., Pauli, P., Vaitl, D., & Stark, R. M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0046782 · Cited by 12 · see citing works
Paelecke–Habermann, Y., Paelecke, M., Mauth, J., Tschisgale, J., Lindenmeyer, J., & Kübler, A. (2019) · Addictive Behaviors Reports · doi:10.1016/j.abrep.2019.100178 · Cited by 11 · see citing works
Heaton, E. C., Seo, E. H., Butkovich, L. M., Yount, S. T., & Gourley, S. L. (2024) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2024.102629 · Cited by 8 · see citing works
Li, W., Chen, X., Luo, Y., Xiao, M., Liu, Y., & Chen, H. (2024) · International Journal of Clinical and Health Psychology · doi:10.1016/j.ijchp.2024.100439 · Cited by 8 · see citing works
Algermissen, J., Rascu, M., Weber, L. A. E., den Boer, T., Martin, E. M., Treeby, B. E., Gray, M. D., Cleveland, R. O., Wittmann, M. K., Clarke, W. T., et al. (2026) · Neuron · doi:10.1016/j.neuron.2026.03.009 · Cited by 7 · see citing works
Computational models of motivated frontal function
O’Reilly, R. C., Russin, J., & Herd, S. A. (2019) · Handbook of clinical neurology · doi:10.1016/b978-0-12-804281-6.00017-3 · Cited by 7 · see citing works
Self-citationModeling Neuromodulation as a Framework to Integrate Uncertainty in General Cognitive Architectures
Alexandre, F., & Carrere, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-41649-6_33 · Cited by 5 · see citing works
Su, Q., Liu, L., Hua, T., Gong, J., Tian, H., Yun, J., & Cai, W. (2025) · Scientific Reports · doi:10.1038/s41598-025-00148-6 · Cited by 1 · see citing works
Wang, N., Xiao, M., Hou, X., Lu, C., Yuan, R., Lam, C., Chen, H., & Cao, B. (2026) · International Journal of Obesity · doi:10.1038/s41366-026-02026-1 · Cited by 1 · see citing works
Modeling pavlovian conditioning with multiple neuronal populations
Carrere, M., & Alexandre, F. (2015) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280716
Computational Link Between Motivational Factors and Cognitive Deficits in Depression
Stolicyn, A., Romaniuk, L., Lawrie, S. M., & Seriès, P. (2026) · Cognitive Computation · doi:10.1101/2025.05.30.25328678
Sleep loss, decision-making, and executive function
Fluke, S. K., Satterfield, B. C., & Killgore, W. D. S. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-13954-3.00028-0
Algermissen, J., Weber, L. A. E., Rascu, M., & Klein-Flügge, M. C. (2025) · doi:10.17605/osf.io/wq7cg
- Authors
- Pauli, W. M., Hazy, T. E., & O'Reilly, R. C.
- Affiliations
- University of Colorado Boulder (US)
- ORCID
- Pauli, Wolfgang M.
- Venue
- Journal of Cognitive Neuroscience
- Publisher
- MIT Press - Journals
- Published
- 2012-02-01
- Volume
- 24
- Issue
- 2
- Pages
- 351-366
- ISSN
- 0898-929X (ISSN-L)
- 1530-8898
- OpenAlex
- W2110416412
- Cited by
- 32
- References
- 117
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 2 · 2020: 3 · 2019: 3 · 2017: 2 · 2016: 3 · 2015: 5 · 2014: 2 · 2013: 3 · 2012: 3
- FWCI
- 1.30
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Cognitive Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health
- Office of Naval Research
- National Institute of Mental Health (P50MH079485, R01MH069597, MH079485, MH069597)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- reward expectancy, orbitofrontal cortex, amygdala, classical conditioning, behavioral flexibility, working memory, neural network model, reversal learning, synaptic plasticity, basolateral amygdala, central amygdala, striatum
- MeSH terms
- Amygdala, Computer Simulation, Conditioning, Psychological, Frontal Lobe, Humans, Models, Neurological, Nerve Net, Neural Pathways, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review