Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia
The DOI matches its Crossref record; 125 of the 125 references it cites were found in a registry or an index.
Abstract
The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and executive functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This article presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia, and amygdala, which together form an actor-critic architecture. The critic system learns which prefrontal representations are task relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model's performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task and other benchmark working memory tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Dahlin et al. (2008) conclude that the observed age differences in plasticity are connected to age-related changes in the striatum”
From Enrichment Effects on Adult Cognitive Development: Can the Functional Capacity of Older Adults Be Preserved and Enhanced? (2008) · cites this paper as O'Reilly & Frank, 2006
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 9,462
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Parallel organization of functionally segregated circuits linking basal ganglia and cortex
Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.09.030186.002041 · Cited by 8,527
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
Self-citationLearning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Learning to forget: Continual prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 4,823
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
An Integrated Theory of the Mind.
Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.1036 · Cited by 2,978
Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex
Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (1989) · Journal of Neurophysiology · doi:10.1152/jn.1989.61.2.331 · Cited by 2,685
THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS
MINK, J. W. (1996) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00042-1 · Cited by 2,486
Simplified neuron model as a principal component analyzer
Oja, E. (1982) · Journal of Mathematical Biology · doi:10.1007/bf00275687 · Cited by 2,309
The phonological loop as a language learning device.
Baddeley, A., Gathercole, S., & Papagno, C. (1998) · Psychological Review · doi:10.1037/0033-295x.105.1.158 · Cited by 2,199
The Atomic Components of Thought
Anderson, J. R., & Lebiere, C. J. (2014) · Psychology Press eBooks · doi:10.4324/9781315805696 · Cited by 2,173
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Cardinal, R. N., Parkinson, J. A., Hall, J., & Everitt, B. J. (2002) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(02)00007-6 · Cited by 2,129
Neuron Activity Related to Short-Term Memory
Fuster, J. M., & Alexander, G. E. (1971) · Science · doi:10.1126/science.173.3997.652 · Cited by 2,085
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Self-citationCircuitry of Primate Prefrontal Cortex and Regulation of Behavior by Representational Memory
Goldman‐Rakic, P. S. (1987) · Comprehensive Physiology · doi:10.1002/cphy.cp010509 · Cited by 1,960
Basal ganglia and cerebellar loops: motor and cognitive circuits
Middleton, F. (2000) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00040-5 · Cited by 1,940
Synaptic depression and cortical gain control
Abbott, L. F., Varela, J. A., Sen, K., & Nelson, S. B. (1997) · Science · doi:10.1126/science.275.5297.221 · Cited by 1,690
Haber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610
Neural Mechanisms of Visual Working Memory in Prefrontal Cortex of the Macaque
Miller, E. K., Erickson, C. A., & Desimone, R. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-16-05154.1996 · Cited by 1,452
Control of Cognitive Processes: Attention and Performance XVIII
Monsell, S., & Driver, J. (2000) · UCL Discovery (University College London) · Cited by 1,451
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens
Nicola, S. M., Surmeier, D. J., & Malenka, R. C. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.185 · Cited by 931
Frank, M. J. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929052880093 · Cited by 925
Self-citationInteractions between frontal cortex and basal ganglia in working memory: A computational model
FRANK, M. J., LOUGHRY, B., & O'REILLY, R. C. (2001) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.1.2.137 · Cited by 879 · cited 9×
“Here we present a summary of our previously developed framework (Frank et al., 2001) for how the BG achieves (cid:1) Robust maintenance occurs via intrinsic PFC mechanisms (bistability, recurrence) in the absence of Go updating signals.”
BackgroundMethodResultInfluentialSelf-citationSynaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory
Wang, X.-J. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-21-09587.1999 · Cited by 859
Neuronal correlate of pictorial short-term memory in the primate temporal cortexYasushi Miyashita
Miyashita, Y., & Chang, H. S. (1988) · Nature · doi:10.1038/331068a0 · Cited by 855
Memory for serial order: A network model of the phonological loop and its timing.
Burgess, N., & Hitch, G. J. (1999) · Psychological Review · doi:10.1037//0033-295x.106.3.551 · Cited by 822
PENNARTZ, C., GROENEWEGEN, H., & LOPESDASILVA, F. (1994) · Progress in Neurobiology · doi:10.1016/0301-0082(94)90025-6 · Cited by 811
Graybiel, A. M., & Ragsdale, C. W. (1978) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.75.11.5723 · Cited by 784
Neuronal activity in monkey ventral striatum related to the expectation of reward
Schultz, W., Apicella, P., Scarnati, E., & Ljungberg, T. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-12-04595.1992 · Cited by 741
Neurocomputational models of working memory
Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000) · Nature Neuroscience · doi:10.1038/81460 · Cited by 724
Prefrontal cortical unit activity and delayed alternation performance in monkeys.
Kubota, K., & Niki, H. (1971) · Journal of Neurophysiology · doi:10.1152/jn.1971.34.3.337 · Cited by 717
Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making
Frank, M. J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.006 · Cited by 706
Self-citationDopamine-Mediated Stabilization of Delay-Period Activity in a Network Model of Prefrontal Cortex
Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.83.3.1733 · Cited by 619
Badre, D., & D'Esposito, M. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.12.2082 · Cited by 615
Joel, D., & Weiner, I. (2000) · Neuroscience · doi:10.1016/s0306-4522(99)00575-8 · Cited by 614 · cited 3×
“…simply being copies of the hidden layer, allowing them to develop in ways that reflect the unique demands of working memory representations (e.g., Rougier & O’Reilly, 2002; Rougier et al., 2005); (3) there can be multiple context layers (i.e., stripes), each with its own set of representations…”
BackgroundSelective Deficits in Prefrontal Cortex Function in Medication-Naive Patients With Schizophrenia
Barch, D. M., Carter, C. S., Braver, T. S., Sabb, F. W., MacDonald, A., Noll, D. C., & Cohen, J. D. (2001) · Archives of General Psychiatry · doi:10.1001/archpsyc.58.3.280 · Cited by 614
Cohen, J. D., Barch, D. M., Carter, C., & Servan-Schreiber, D. (1999) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.108.1.120 · Cited by 612
Houk, J. C., & Wise, S. P. (1995) · Cerebral Cortex · doi:10.1093/cercor/5.2.95 · Cited by 593
Dissociating working memory from task difficulty in human prefrontal cortex
Barch, D. M., Braver, T. S., Nystrom, L. E., Forman, S. D., Noll, D. C., & Cohen, J. D. (1997) · Neuropsychologia · doi:10.1016/s0028-3932(97)00072-9 · Cited by 590
Frank, M. J., & Claus, E. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.300 · Cited by 585
Self-citationA Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement
Houk, J. C., Adams, J. L., & Barto, A. G. (1994) · Models of Information Processing in the Basal Ganglia · doi:10.7551/mitpress/4708.003.0020 · Cited by 541
Braver, T. S., Barch, D. M., Keys, B. A., Carter, C. S., Cohen, J. D., Kaye, J. A., Janowsky, J. S., Taylor, S. F., Yesavage, J. A., Mumenthaler, M. S., et al. (2001) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.130.4.746 · Cited by 535
Cowan, R. L., & Wilson, C. J. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.71.1.17 · Cited by 530
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Hernández-López, S., Bargas, J., Surmeier, D. J., Reyes, A., & Galarraga, E. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-09-03334.1997 · Cited by 495
Cohen, J. D., Braver, T. S., & O´Reilly, R. (1996) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1996.0138 · Cited by 487
Self-citationFrank, M. J., & O'Reilly, R. C. (2006) · Behavioral Neuroscience · doi:10.1037/0735-7044.120.3.497 · Cited by 472
Self-citationSchizophrenic Deficits in the Processing of Context: A Test of a Theoretical Model
Servan-Schreiber, D. (1996) · Archives of General Psychiatry · doi:10.1001/archpsyc.1996.01830120037008 · Cited by 462
Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2055 · Cited by 452
Self-citationHernández-López, S., Tkatch, T., Perez-Garci, E., Galarraga, E., Bargas, J., Hamm, H., & Surmeier, D. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-24-08987.2000 · Cited by 443
A role for NMDA-receptor channels in working memory
Lisman, J. E., Fellous, J.-M., & Wang, X.-J. (1998) · Nature Neuroscience · doi:10.1038/1086 · Cited by 438
Gradient-based learning algorithms for recurrent networks and their computational complexity
Williams, R. J., & Zipser, D. (1995) · Cited by 424
The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions
Bogacz, R., & Gurney, K. (2007) · Neural Computation · doi:10.1162/neco.2007.19.2.442 · Cited by 417
On the Control of Control: The Role of Dopamine in Regulating Prefrontal Function and Working Memory
(2000) · Control of Cognitive Processes · doi:10.7551/mitpress/1481.003.0044 · Cited by 416
The Role of Frontopolar Cortex in Subgoal Processing during Working Memory
Braver, T. S., & Bongiolatti, S. R. (2002) · NeuroImage · doi:10.1006/nimg.2001.1019 · Cited by 409
(2013) · doi:10.4324/9780203763247 · Cited by 400
Prefrontal cortex and flexible cognitive control: Rules without symbols
Rougier, N. P., Noelle, D. C., Braver, T. S., Cohen, J. D., & O'Reilly, R. C. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0502455102 · Cited by 398
Self-citationStern, E. A., Kincaid, A. E., & Wilson, C. J. (1997) · Journal of Neurophysiology · doi:10.1152/jn.1997.77.4.1697 · Cited by 361
O'Reilly, R. C. (1996) · Neural Computation · doi:10.1162/neco.1996.8.5.895 · Cited by 352
Self-citationThe role of inner speech in task switching: A dual-task investigation
Emerson, M. J., & Miyake, A. (2003) · Journal of Memory and Language · doi:10.1016/s0749-596x(02)00511-9 · Cited by 348
Molecular effects of dopamine on striatal-projection pathways
Gerfen, C. R. (2000) · Trends in Neurosciences · doi:10.1016/s1471-1931(00)00019-7 · Cited by 341
Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning
Koechlin, E., Corrado, G., Pietrini, P., & Grafman, J. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.130177397 · Cited by 330
Waltz, J. A., Frank, M. J., Robinson, B. M., & Gold, J. M. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.09.042 · Cited by 327
Self-citationA Biologically Based Computational Model of Working Memory
O'Reilly, R. C., Braver, T. S., & Cohen, J. D. (1999) · Models of Working Memory · doi:10.1017/cbo9781139174909.014 · Cited by 326
Self-citationSix principles for biologically based computational models of cortical cognition
O'Reilly, R. C. (1998) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01241-8 · Cited by 318
Self-citationKelley, A. E., Smith-Roe, S. L., & Holahan, M. R. (1997) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.94.22.12174 · Cited by 293
Lewis, B. L. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.12.1168 · Cited by 287
Plenz, D., & Kitai, S. T. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-01-00266.1998 · Cited by 278
Morton, J. B., & Munakata, Y. (2002) · Developmental Psychobiology · doi:10.1002/dev.10033 · Cited by 260
Levitt, J. B., Lewis, D. A., Yoshioka, T., & Lund, J. S. (1993) · Journal of Comparative Neurology · doi:10.1002/cne.903380304 · Cited by 259
Neurobiology of Decision-Making
(1996) · Research and Perspectives in Neurosciences · doi:10.1007/978-3-642-79928-0 · Cited by 253
O'Reilly, R. C. (2002) · Cerebral Cortex · doi:10.1093/cercor/12.3.246 · Cited by 241
Self-citationSeparate neural substrates for skill learning and performance in the ventral and dorsal striatum
Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., & O'Reilly, R. C. (2007) · Nature Neuroscience · doi:10.1038/nn1817 · Cited by 241
Self-citationModel of Cortical-Basal Ganglionic Processing: Encoding the Serial Order of Sensory Events
Beiser, D. G., & Houk, J. C. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.79.6.3168 · Cited by 238
A Computational Model of How the Basal Ganglia Produce Sequences
Berns, G. S., & Sejnowski, T. J. (1998) · Journal of Cognitive Neuroscience · doi:10.1162/089892998563815 · Cited by 232
Surround inhibition among projection neurons is weak or nonexistent in the rat neostriatum
Jaeger, D., Kita, H., & Wilson, C. J. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.5.2555 · Cited by 230
Frank, M. J., Santamaria, A., O'Reilly, R. C., & Willcutt, E. (2007) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301278 · Cited by 227
A Neurocomputational Theory of the Dopaminergic Modulation of Working Memory Functions
Durstewitz, D., Kelc, M., & Güntürkün, O. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-07-02807.1999 · Cited by 225
Basal ganglia and dopamine contributions to probabilistic category learning
Shohamy, D., Myers, C. E., Kalanithi, J., & Gluck, M. A. (2008) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2007.07.008 · Cited by 218
Deterministic Boltzmann Learning Performs Steepest Descent in Weight-Space
Hinton, G. E. (1989) · Neural Computation · doi:10.1162/neco.1989.1.1.143 · Cited by 206
Patterns of intrinsic and associational circuitry in monkey prefrontal cortex
Pucak, M. L., Levitt, J. B., Lund, J. S., & Lewis, D. A. (1996) · The Journal of Comparative Neurology · doi:10.1002/(sici)1096-9861(19961223)376:4<614::aid-cne9>3.0.co;2-4 · Cited by 203
Action selection and refinement in subcortical loops through basal ganglia and cerebellum
Houk, J. C., Bastianen, C., Fansler, D., Fishbach, A., Fraser, D., Reber, P. J., Roy, S. A., & Simo, L. S. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2063 · Cited by 197
Nakahara, H., Doya, K., & Hikosaka, O. (2001) · Journal of Cognitive Neuroscience · doi:10.1162/089892901750363208 · Cited by 191
A Model of Corticostriatal Plasticity for Learning Oculomotor Associations and Sequences
Dominey, P., Arbib, M., & Joseph, J.-P. (1995) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1995.7.3.311 · Cited by 190
Early consolidation of instrumental learning requires protein synthesis in the nucleus accumbens
Hernandez, P. J., Sadeghian, K., & Kelley, A. E. (2002) · Nature Neuroscience · doi:10.1038/nn973 · Cited by 186
Temporal dynamics of basal ganglia response and connectivity during verbal working memory
Chang, C., Crottaz-Herbette, S., & Menon, V. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.08.056 · Cited by 172
Frank, M. J., Scheres, A., & Sherman, S. J. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2058 · Cited by 166
Self-citationSelection of actions in the basal ganglia–thalamocortical circuits: review and model
Kropotov, J. D., & Etlinger, S. C. (1999) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(98)00051-8 · Cited by 165
Gorelova, N. A., & Yang, C. R. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.1.75 · Cited by 162
(1993) · Elsevier eBooks · doi:10.1016/c2009-0-35344-1 · Cited by 161
PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm.
O’Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007) · Behavioral Neuroscience · doi:10.1037/0735-7044.121.1.31 · Cited by 160
Self-citationA Computational Model of Information Processing in the Frontal Cortex and Basal Ganglia
Amos, A. J. (2000) · Journal of Cognitive Neuroscience · doi:10.1162/089892900562174 · Cited by 160
Reward-related Signals Carried by Dopamine Neurons
Schultz, W., Romo, R., Ljungberg, T., Mirenowicz, J., Hollerman, J. R., & Dickinson, A. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0019 · Cited by 154
Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning
O’Reilly, R. C. (2001) · Neural Computation · doi:10.1162/08997660152002834 · Cited by 149
Self-citationContrastive Hebbian Learning in the Continuous Hopfield Model
Movellan, J. R. (1991) · Elsevier eBooks · doi:10.1016/b978-1-4832-1448-1.50007-x · Cited by 132
Pre- and postsynaptic localization of GABAB receptors in the basal ganglia in monkeys
Charara, A., Heilman, C. J., Levey, A. I., & Smith, Y. (1999) · Neuroscience · doi:10.1016/s0306-4522(99)00409-1 · Cited by 131
Miller, G. (2004) · Science · doi:10.1126/science.304.5667.34 · Cited by 130
Modeling functions of striatal dopamine modulation in learning and planning
Suri, R. E., Bargas, J., & Arbib, M. A. (2001) · Neuroscience · doi:10.1016/s0306-4522(00)00554-6 · Cited by 128
Surmeier, D. J., & Kitai, S. T. (1993) · Progress in brain research · doi:10.1016/s0079-6123(08)61354-0 · Cited by 123
Multilevel structure in behaviour and in the brain: a model of Fuster's hierarchy
Botvinick, M. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2056 · Cited by 118
A Predictive Reinforcement Model of Dopamine Neurons for Learning Approach Behavior
Contreras-Vidal, J. L., & Schultz, W. (1999) · Journal of Computational Neuroscience · doi:10.1023/a:1008862904946 · Cited by 106
How the Basal Ganglia Make Decisions
Berns, G. S., & B, T. J. S. A. (1996) · Research and perspectives in neurosciences · doi:10.1007/978-3-642-79928-0_6 · Cited by 94
Learning representations in a gated prefrontal cortex model of dynamic task switching
Rougier, N. P., & O’Reilly, R. C. (2002) · Cognitive Science · doi:10.1207/s15516709cog2604_4 · Cited by 84
Self-citationAdaptive Neural Networks in the Basal Ganglia
Graybiel, A. M., & Kimura, M. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0010 · Cited by 75
A spiking neuron model of cortical broadcast and competition
Shanahan, M. (2007) · Consciousness and Cognition · doi:10.1016/j.concog.2006.12.005 · Cited by 75
Qin, S., Piekema, C., Petersson, K. M., Han, B., Luo, J., & Fernández, G. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.07.020 · Cited by 70
Introduction. Modelling natural action selection
Prescott, T. J., Bryson, J. J., & Seth, A. K. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2050 · Cited by 66
Effects of local connectivity on striatal function: Simulation and analysis of a model
Wickens, J. R., Kötter, R., & Alexander, M. E. (1995) · Synapse · doi:10.1002/syn.890200402 · Cited by 62
Load response functions in the human spatial working memory circuit during location memory updating
Leung, H., Oh, H., Ferri, J. M., & Yi, Y. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.12.012 · Cited by 61
Sensorimotor Selection and the Basal Ganglia: A Neural Network Model
Houk, J. C., Davis, J. L., & Beiser, D. G. (1994) · Cited by 42
The Prefrontal Cortex: Anatomy, Physiology and Neuropsychology of the Frontal Lobe, 3rd Edition.
Mayer, R. F. (1999) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199902000-00012 · Cited by 38
Learning Unambiguous Reduced Sequence Descriptions
Schmidhuber, J. (1991) · Cited by 33
A Model of the Phonological Loop: Generalization and Binding
O’Reilly, R. C., & Soto, R. (2002) · Defense Technical Information Center (DTIC) · doi:10.21236/ada496081 · Cited by 19
Self-citationA neural network model of hippocampal–striatal–prefrontal interactions in contextual conditioning
Turnock, M., & Becker, S. (2007) · Brain Research · doi:10.1016/j.brainres.2007.06.078 · Cited by 18
Dilmore, J. G., Gutkin, B. S., & Ermentrout, B. (1999) · Neurocomputing · doi:10.1016/s0925-2312(99)00005-3 · Cited by 11
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Miyake, A., & Friedman, N. P. (2012) · Current Directions in Psychological Science · doi:10.1177/0963721411429458 · Cited by 3,944
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
Habits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697
Metadata
- Authors
- O'Reilly, R. C., & Frank, M. J.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2006-02-01
- Volume
- 18
- Issue
- 2
- Pages
- 283-328
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2111534506
- Cited by
- 1,066
- References
- 125
- FWCI
- 15.56
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.