Goal-directed control and its antipodes
Abstract
In instrumental conditioning, there is a rather precise definition of goal-directed control, and therefore an acute boundary between it and the somewhat more amorphous category comprising its opposites. Here, we review this division in terms of the various distinctions that accompany it in the fields of reinforcement learning and cognitive architectures, considering issues such as declarative and procedural control, the effect of prior distributions over environments, the neural substrates involved, and the differing views about the relative rationality of the various forms of control. Our overall aim is to reconnect some presently far-flung relations.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Contemporary associative learning theories assume that action selection depends on the continuous cooperation and competition of several behavioural controllers, divided into Pavlovian and instrumental (goal-directed and habitual).
“Contemporary associative learning theories assume that action selection depends on the continuous cooperation and competition of several behavioural controllers, which can be divided into “Pavlovian” and “instrumental” (goal-directed and habitual) (Balleine and Dickinson, 1998, Daw et al., 2005, Dayan, 2009).”
From Active Inference, homeostatic regulation and adaptive behavioural control (2015) · cites this paper as Dayan, 2009
Habitual instrumental control is characterized by automaticity, computational efficiency, and inflexibility, while goal-directed control is characterized by active deliberation, high computational cost, and adaptive flexibility to changing environmental contingencies.
“Thus, key characteristics of habitual instrumental control include automaticity, computational efficiency, and inflexibility, while characteristics of goal-directed control include active deliberation, high computational cost, and an adaptive flexibility to changing environmental contingencies (Dayan, 2009).”
From Goals and Habits in the Brain (2013) · cites this paper as Dayan, 2009
Cites
Artificial intelligence: a modern approach
Kumar, D. A., Balasubramanian, S., Byeon, D. H., & Manerkar, P. G. V. (1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 22,182 · see citing works
(1995) · Cited by 10,558 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Self-citationSimple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
Distributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783 · see citing works
(1999) · Cited by 6,104 · see citing works
Controlled and automatic human information processing: I. Detection, search, and attention.
Schneider, W., & Shiffrin, R. M. (1977) · Psychological Review · doi:10.1037/0033-295x.84.1.1 · Cited by 5,902 · see citing works
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,406 · see citing works
The Behavior of Organisms: An Experimental Analysis
Jones, F. N., & Skinner, B. F. (1939) · The American Journal of Psychology · doi:10.2307/1416495 · Cited by 4,388 · see citing works
Individual differences in reasoning: Implications for the rationality debate?
Stanovich, K. E., & West, R. F. (2000) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00003435 · Cited by 4,255 · see citing works
(1991) · Choice Reviews Online · doi:10.5860/choice.28-5376 · Cited by 4,217 · see citing works
A capacity theory of comprehension: individual differences in working memory
Just, M. A., & Carpenter, P. A. (1992) · Psychological Review · doi:10.1037/0033-295x.99.1.122 · Cited by 4,153 · see citing works
The empirical case for two systems of reasoning.
Sloman, S. A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.1.3 · Cited by 3,813 · see citing works
Acquisition of cognitive skill
Anderson, J. R. (1982) · Psychological Review · doi:10.1037/0033-295x.89.4.369 · Cited by 3,663 · see citing works
Heuristics and Biases: The Psychology of Intuitive Judgment
Bottom, W. P., Gilovich, T. D., Griffin, D., & Kahneman, D. J. (2004) · Academy of Management Review · doi:10.2307/20159081 · Cited by 3,579 · see citing works
An internal model for sensorimotor integration
Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995) · Science · doi:10.1126/science.7569931 · Cited by 3,553 · see citing works
Representativeness revisited: Attribute substitution in intuitive judgment.
Kahneman, D., & Frederick, S. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.004 · Cited by 3,476 · see citing works
Toward an instance theory of automatization.
Logan, G. D. (1988) · Psychological Review · doi:10.1037/0033-295x.95.4.492 · Cited by 3,466 · see citing works
Preface: Cerebral Cortex Has Come of Age
Goldman, P. S., & RAKIC, P. T. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1-a · Cited by 3,412 · see citing works
Some Philosophical Problems from the Standpoint of Artificial Intelligence
McCarthy, J., & Hayes, P. J. (1981) · Elsevier eBooks · doi:10.1016/b978-0-934613-03-3.50033-7 · Cited by 3,296 · see citing works
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246 · see citing works
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119 · see citing works
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · see citing works
An Integrated Theory of the Mind.
Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C. J., & Qin, Y. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.1036 · Cited by 3,048 · see citing works
Cognitive Skills and Their Acquisition
Anderson, J. R. (2013) · Psychology Press eBooks · doi:10.4324/9780203728178 · Cited by 2,926 · see citing works
Integration of the cognitive and the psychodynamic unconscious
Epstein, S. (1994) · American Psychologist · doi:10.1037/0003-066x.49.8.709 · Cited by 2,883 · cited 1× · see citing works
“…famous heuristics and biases program (Tversky & Kahneman, 1971), and indeed a wealth of other dual process theories (e.g., Chaiken and Trope (1999), Epstein (1994), Evans (2003), Schneider and Shiffrin (1977), Shiffrin and Schneider (1984), Sloman (1996), Stanovich andWest (2002)) is the notion…”
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
Self-citationLearned helplessness: Theory and evidence
Maier, S. F., & Seligman, M. E. (1976) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.105.1.3 · Cited by 2,494 · see citing works
Models of Working Memory: Mechanisms of Active Maintenance and Executive Control
Miyake, A., & Shah, P. (1999) · Cited by 2,404 · see citing works
Memory systems of the brain: A brief history and current perspective
Squire, L. R. (2004) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2004.06.005 · Cited by 2,269 · see citing works
In two minds: dual-process accounts of reasoning
Evans, J. S. B. T. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.08.012 · Cited by 2,252 · see citing works
The Atomic Components of Thought
Anderson, J. R., & Lebiere, C. J. (2014) · Psychology Press eBooks · doi:10.4324/9781315805696 · Cited by 2,181 · see citing works
Multiple paired forward and inverse models for motor control
Wolpert, D. M., & Kawato, M. (1998) · Neural Networks · doi:10.1016/s0893-6080(98)00066-5 · Cited by 2,152 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Self-citationThe Architecture of Cognitive Control in the Human Prefrontal Cortex
Koechlin, E., Ody, C., & Kouneiher, F. (2003) · Science · doi:10.1126/science.1088545 · Cited by 1,922 · see citing works
The neural correlates of subjective value during intertemporal choice
Kable, J. W., & Glimcher, P. W. (2007) · Nature Neuroscience · doi:10.1038/nn2007 · Cited by 1,916 · see citing works
Artificial Intelligence: A Modern Approach
Russell, S., & Norvig, P. (1995) · Cited by 1,753 · see citing works
Reverse replay of behavioural sequences in hippocampal place cells during the awake state
Foster, D. J., & Wilson, M. A. (2006) · Nature · doi:10.1038/nature04587 · Cited by 1,727 · see citing works
Decisions from Experience and the Effect of Rare Events in Risky Choice
Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004) · Psychological Science · doi:10.1111/j.0956-7976.2004.00715.x · Cited by 1,622 · see citing works
Species-specific defense reactions and avoidance learning
Bolles, R. C. (1970) · Psychological Review · doi:10.1037/h0028589 · Cited by 1,610 · see citing works
Haber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610 · see citing works
Modulation of memory fields by dopamine Dl receptors in prefrontal cortex
Williams, G. V., & Goldman-Rakic, P. S. (1995) · Nature · doi:10.1038/376572a0 · Cited by 1,593 · see citing works
Mechanisms of skill acquisition and the law of practice
Newell, A., & Rosenbloom, P. S. (2018) · Figshare · doi:10.1184/r1/6607196 · Cited by 1,501 · see citing works
Control of Cognitive Processes: Attention and Performance XVIII
Monsell, S., & Driver, J. (2000) · UCL Discovery (University College London) · Cited by 1,451 · see citing works
Meyer, D. E., & Kieras, D. E. (1997) · Psychological Review · doi:10.1037/0033-295x.104.1.3 · Cited by 1,401 · see citing works
Memory of sequential experience in the hippocampus during slow wave sleep
Lee, A. K., & Wilson, M. A. (2002) · Neuron · doi:10.1016/s0896-6273(02)01096-6 · Cited by 1,296 · see citing works
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217 · see citing works
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207 · see citing works
Self-citationForward and reverse hippocampal place-cell sequences during ripples
Diba, K., & Buzsáki, G. (2007) · Nature Neuroscience · doi:10.1038/nn1961 · Cited by 1,205 · see citing works
Breland, K., & Breland, M. (1961) · American Psychologist · doi:10.1037/h0040090 · Cited by 1,192 · see citing works
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190 · see citing works
An information theoretical approach to prefrontal executive function
Koechlin, E., & Summerfield, C. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.04.005 · Cited by 1,183 · see citing works
Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience
Skaggs, W. E., & McNaughton, B. L. (1996) · Science · doi:10.1126/science.271.5257.1870 · Cited by 1,173 · see citing works
Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep
Louie, K., & Wilson, M. A. (2001) · Neuron · doi:10.1016/s0896-6273(01)00186-6 · Cited by 1,148 · see citing works
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · see citing works
Self-citationThe "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070 · see citing works
Self-citationO’Reilly, R. C., & Frank, M. J. (2005) · Neural Computation · doi:10.1162/089976606775093909 · Cited by 1,066 · see citing works
Neuroeconomics: decision making and the brain
(2014) · Choice Reviews Online · doi:10.5860/choice.51-5702 · Cited by 1,020 · see citing works
5-HT and mechanisms of defence
Deakin, J. F. W., & Graeff, F. G. (1991) · Journal of Psychopharmacology · doi:10.1177/026988119100500414 · Cited by 1,011 · see citing works
Melton, A. W. (1964) · Elsevier eBooks · doi:10.1016/c2013-0-12392-6 · Cited by 985 · see citing works
Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system
Lammel, S., Hetzel, A., Häckel, O., Jones, I., Liss, B., & Roeper, J. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.022 · Cited by 976 · see citing works
Interactions between frontal cortex and basal ganglia in working memory: A computational model
FRANK, M. J., LOUGHRY, B., & O'REILLY, R. C. (2001) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.1.2.137 · Cited by 879 · see citing works
Hammond, K. R. (1996) · doi:10.1093/oso/9780195097344.001.0001 · Cited by 851 · see citing works
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · see citing works
Self-citationThe orbitofrontal cortex and beyond: From affect to decision-making
Rolls, E. T., & Fabian Grabenhorst, B. (2008) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2008.09.001 · Cited by 808 · see citing works
Dyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777 · cited 2× · see citing works
“…(‘dreamt’) by the generative model can be used during times (notably sleeping, grooming and eating for rats, Diba & Buzsáki, 2007; Foster & Wilson, 2006; Lee & Wilson, 2002; Louie &Wilson, 2001; Skaggs &Mcnaughton, 1996) that active control is suppressed (Hinton et al., 1995; Sutton, 1991).”
Anterior Prefrontal Function and the Limits of Human Decision-Making
Koechlin, É., & Hyafil, A. (2007) · Science · doi:10.1126/science.1142995 · Cited by 750 · see citing works
Coordination of Actions and Habits in the Medial Prefrontal Cortex of Rats
Killcross, S. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.4.400 · Cited by 748 · see citing works
AUTO‐MAINTENANCE IN THE PIGEON: SUSTAINED PECKING DESPITE CONTINGENT NON‐REINFORCEMENT2
Williams, D. R., & Williams, H. G. (1969) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1969.12-511 · Cited by 717 · see citing works
(1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.3.301 · Cited by 711 · see citing works
Belin, D., & Everitt, B. J. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.019 · Cited by 697 · see citing works
Improving generalization for temporal difference learning: The successor representation
Dayan, P. (1993) · Neural Computation · doi:10.1162/neco.1993.5.4.613 · Cited by 693 · see citing works
Self-citationNeuroeconomics: Decision Making and the Brain
Rushworth, M. F. S., & Walton, M. E. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.005 · Cited by 669 · see citing works
Fitts, P. M. (1964) · Elsevier eBooks · doi:10.1016/b978-1-4832-3145-7.50016-9 · Cited by 662 · see citing works
Parallel incentive processing: an integrated view of amygdala function
Balleine, B. W., & Killcross, S. (2006) · Trends in Neurosciences · doi:10.1016/j.tins.2006.03.002 · Cited by 622 · see citing works
Badre, D., & D'Esposito, M. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.12.2082 · Cited by 615 · see citing works
Joel, D., & Weiner, I. (2000) · Neuroscience · doi:10.1016/s0306-4522(99)00575-8 · Cited by 614 · see citing works
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597 · see citing works
STEVENS' HANDBOOK OF EXPERIMENTAL PSYCHOLOGY
Pashler, H. (2002) · Wiley eBooks · doi:10.1002/0471214426 · Cited by 576 · see citing works
Determining the neural substrates of goal-directed learning in the human brain
Valentin, V. V., Dickinson, A., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0564-07.2007 · Cited by 529 · see citing works
A THEORY OF THE ACQUISITION OF SPEED-SKILL∗
CROSSMAN, E. R. F. W. (1959) · Ergonomics · doi:10.1080/00140135908930419 · Cited by 517 · see citing works
Controlled and Automatic Human Information Processing: 1. Detection, Search, and Attention.
Schneider, W., & Shiffrin, R. M. (1977) · Psychological Review · Cited by 506 · see citing works
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503 · see citing works
Melton, A. W. (1964) · Defense Technical Information Center (DTIC) · Cited by 501 · see citing works
Adaptive Critics and the Basal Ganglia
Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0018 · Cited by 466 · see citing works
On the Control of Control: The Role of Dopamine in Regulating Prefrontal Function and Working Memory
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/1481.003.0044 · Cited by 442 · see citing works
Reynolds, S. M., & Berridge, K. C. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-16-07308.2002 · Cited by 403 · see citing works
Prefrontal cortex and flexible cognitive control: Rules without symbols
Rougier, N. P., Noelle, D. C., Braver, T. S., Cohen, J. D., & O’Reilly, R. C. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0502455102 · Cited by 397 · see citing works
The misbehavior of value and the discipline of the will
Dayan, P., Niv, Y., Seymour, B., & Daw, N. D. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.002 · Cited by 389 · see citing works
Self-citationProbabilistic Models of the Brain : Perception and Neural Function
Rao, R. P. N., Olshausen, B. A., & Lewicki, M. S. (2002) · Cited by 385 · see citing works
Serotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381 · see citing works
Self-citationLearning and Sequential Decision Making
Barto, A. G., Sutton, R. S., & Watkins, C. (1989) · Cited by 349 · see citing works
Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits
BALLEINE, B. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.08.061 · Cited by 326 · see citing works
A Biologically Based Computational Model of Working Memory
O’Reilly, R. C., Braver, T. S., & Cohen, J. D. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9781139174909.014 · Cited by 323 · see citing works
Internal Models for Motor Control
Kawato, M., & Wolpert, D. M. (2007) · Novartis Foundation symposium · doi:10.1002/9780470515563.ch16 · Cited by 323 · see citing works
March, J. G. (1996) · Psychological Review · doi:10.1037/0033-295x.103.2.309 · Cited by 307 · see citing works
Human Pavlovian–Instrumental Transfer
Talmi, D., Seymour, B., Dayan, P., & Dolan, R. J. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.4028-07.2008 · Cited by 302 · see citing works
Self-citationSuri, R. E., & Schultz, W. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00697-6 · Cited by 294 · see citing works
Automatic and controlled processing revisited.
Shiffrin, R. M., & Schneider, W. (1984) · Psychological Review · doi:10.1037/0033-295x.91.2.269 · Cited by 283 · see citing works
Reynolds, S. M., & Berridge, K. C. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-09-03261.2001 · Cited by 281 · see citing works
Calculating Consequences: Brain Systems That Encode the Causal Effects of Actions
Tanaka, S. C., Balleine, B. W., & O'Doherty, J. P. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1808-08.2008 · Cited by 268 · see citing works
Frames and brains: elicitation and control of response tendencies
Kahneman, D., & Frederick, S. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.11.007 · Cited by 260 · see citing works
Reynolds, S. M., & Berridge, K. C. (2008) · Nature Neuroscience · doi:10.1038/nn2061 · Cited by 244 · see citing works
Flexible shaping: How learning in small steps helps
Krueger, K. A., & Dayan, P. (2009) · Cognition · doi:10.1016/j.cognition.2008.11.014 · Cited by 221 · see citing works
Self-citationO’Doherty, J. P. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1401.036 · Cited by 201 · see citing works
Todorov, E. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/1535.003.0018 · Cited by 198 · see citing works
Neural blackboard architectures of combinatorial structures in cognition
van der Velde, F., & de Kamps, M. (2006) · Behavioral and Brain Sciences · doi:10.1017/s0140525x06009022 · Cited by 195 · see citing works
The Neural Mechanisms Underlying the Influence of Pavlovian Cues on Human Decision Making
Bray, S., Rangel, A., Shimojo, S., Balleine, B., & O'Doherty, J. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.0897-08.2008 · Cited by 193 · see citing works
The atomic components of thought
Taatgen, N. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01250-9 · Cited by 184 · see citing works
Why do children learn to say “Broke”? A model of learning the past tense without feedback
Taatgen, N. A., & Anderson, J. R. (2002) · Cognition · doi:10.1016/s0010-0277(02)00176-2 · Cited by 175 · see citing works
From recurrent choice to skill learning: A reinforcement-learning model.
Fu, W., & Anderson, J. R. (2006) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.135.2.184 · Cited by 165 · see citing works
Niv, Y., Joel, D., Meilijson, I., & Ruppin, E. (2002) · Adaptive Behavior · doi:10.1177/10597123020101001 · Cited by 157 · see citing works
A forward-inverse optics model of reciprocal connections between visual cortical areas
Kawato, M., Hayakawa, H., & Inui, T. (1993) · Network: Computation in Neural Systems · doi:10.1088/0954-898x_4_4_001 · Cited by 151 · see citing works
Anderson, J. R., Fincham, J. M., Qin, Y., & Stocco, A. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.01.006 · Cited by 146 · see citing works
TD Models: Modeling the World at a Mixture of Time Scales
Sutton, R. S. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50072-4 · Cited by 144 · see citing works
A Bayesian formulation of behavioral control
Huys, Q. J. M., & Dayan, P. (2009) · Cognition · doi:10.1016/j.cognition.2009.01.008 · Cited by 144 · see citing works
Self-citationAchieving master level play in 9×9 computer go
Gelly, S., & Silver, D. (2008) · UCL Discovery (University College London) · Cited by 119 · see citing works
The acquisition of robust and flexible cognitive skills.
Taatgen, N. A., Huss, D., Dickison, D., & Anderson, J. R. (2008) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.137.3.548 · Cited by 115 · see citing works
Values and Actions in Aversion
Dayan, P., & Seymour, B. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00012-9 · Cited by 71 · see citing works
Self-citationBilinearity, rules, and prefrontal cortex
Dayan, P. (2007) · Frontiers in Computational Neuroscience · doi:10.3389/neuro.10.001.2007 · Cited by 66 · see citing works
Self-citationThe Misbehaviour of Value and the Discipline of the Will
Dayan, P. (2006) · Cited by 60 · see citing works
Self-citationFu, W., & Anderson, J. R. (2007) · Psychological Research · doi:10.1007/s00426-007-0113-7 · Cited by 52 · see citing works
Reinforcers and control : towards a computational aetiology of depression
Huys, Q. J. M. (2007) · UCL Discovery (University College London) · Cited by 16 · see citing works
Recurrent Sampling Models for the Helmholtz Machine
Dayan, P. (1999) · Neural Computation · doi:10.1162/089976699300016610 · Cited by 15 · see citing works
Self-citationXiong (1999) · doi:10.1002/(sici)1097-0193(1999)8:2/3
Flagged first, then by citations
6 of the 131 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977 · see citing works
Self-citationActive Inference, homeostatic regulation and adaptive behavioural control
Pezzulo, G., Rigoli, F., & Friston, K. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.001 · Cited by 650 · see citing works
“Contemporary associative learning theories assume that action selection depends on the continuous cooperation and competition of several behavioural controllers, which can be divided into ‘‘Pavlovian’’ and ‘‘instrumental’’ (goal-directed and habitual) (Balleine and Dickinson, 1998; Daw et al., 2005; Dayan, 2009).”
InfluentialHuang, V. S., Haith, A., Mazzoni, P., & Krakauer, J. W. (2011) · Neuron · doi:10.1016/j.neuron.2011.04.012 · Cited by 472 · see citing works
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J. R., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 430 · see citing works
Otto (2013) · doi:10.1177/0956797612463080 · Cited by 352 · see citing works
Model-Based and Model-Free Mechanisms of Human Motor Learning
Haith, A. M., & Krakauer, J. W. (2013) · Advances in Experimental Medicine and Biology · doi:10.1007/978-1-4614-5465-6_1 · Cited by 263 · see citing works
The Cognitive Architecture of Spatial Navigation: Hippocampal and Striatal Contributions
Chersi, F., & Burgess, N. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.021 · Cited by 240 · see citing works
Neuromodulation of reward‐based learning and decision making in human aging
Eppinger, B., Hämmerer, D., & Li, S. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06230.x · Cited by 228 · see citing works
Reinforcement learning in depression: a review of computational research
Chen, C., Takahashi, T., Nakagawa, S., Inoue, T., & Kusumi, I. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.05.005 · Cited by 233 · see citing works
The Influence of Movement Preparation Time on the Expression of Visuomotor Learning and Savings
Haith, A. M., Huberdeau, D. M., & Krakauer, J. W. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3869-14.2015 · Cited by 220 · see citing works
The multiple effects of practice: skill, habit and reduced cognitive load
Haith, A. M., & Krakauer, J. W. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.01.015 · Cited by 206 · see citing works
Solway, A., & Botvinick, M. (2012) · Psychological Review · doi:10.1037/a0026435 · Cited by 180 · see citing works
Observing the observer (I): meta-bayesian models of learning and decision-making
Daunizeau, J., den Ouden, H. E. M., Pessiglione, M., Kiebel, S. J., Stephan, K. E., & Friston, K. J. (2010) · PLoS ONE · doi:10.1371/journal.pone.0015554 · Cited by 167 · see citing works
Shine, J. M. (2020) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2020.101951 · Cited by 159 · see citing works
Pezzulo, G., Rigoli, F., & Chersi, F. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00092 · Cited by 163 · see citing works
Dual-process decomposition in human sensorimotor adaptation
Huberdeau, D. M., Krakauer, J. W., & Haith, A. M. (2015) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2015.03.003 · Cited by 161 · see citing works
Beyond dichotomies in reinforcement learning
Collins, A. G. E., & Cockburn, J. (2020) · Nature reviews. Neuroscience · doi:10.1038/s41583-020-0355-6 · Cited by 128 · see citing works
A critical review of the most popular types of neuro control
Mohammadzaheri, M., Chen, L., & Grainger, S. (2011) · Asian Journal of Control · doi:10.1002/asjc.449 · Cited by 115 · see citing works
Mattar, M. G., & Lengyel, M. (2022) · Neuron · doi:10.1016/j.neuron.2021.12.018 · Cited by 113 · see citing works
Chatila, R., Renaudo, E., Andries, M., Chavéz-García, R. O., Luce-Vayrac, P., Gottstein, R., Alami, R., Clodic, A., Devin, S., Girard, B., et al. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00088 · Cited by 106 · see citing works
How to set the switches on this thing
Dayan, P. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.011 · Cited by 99 · see citing works
Self-citationIntelligent problem-solving as integrated hierarchical reinforcement learning
Eppe, M., Gumbsch, C., Kerzel, M., Nguyen, P. D. H., Butz, M. V., & Wermter, S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-021-00433-9 · Cited by 98 · see citing works
Open Problems in Cooperative AI
Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., & Graepel, T. (2020) · arXiv · doi:10.48550/arxiv.2012.08630 · Cited by 93 · see citing works
Modeling Habit Patterns Using Conditional Reflexes in Agency
Khan, Q. H., Ghazal, T. M., Abbas, S., Khan, W. A., Khan, M. A., Said, R. A., Ahmad, M., & Asif, M. K. (2021) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2021.018888 · Cited by 93 · see citing works
How Outcome Uncertainty Mediates Attention, Learning, and Decision-Making
Monosov, I. E. (2020) · Trends in Neurosciences · doi:10.1016/j.tins.2020.06.009 · Cited by 87 · see citing works
Model-based action planning involves cortico-cerebellar and basal ganglia networks
Fermin, A. S. R., Yoshida, T., Yoshimoto, J., Ito, M., Tanaka, S., & Doya, K. (2016) · Scientific Reports · doi:10.1038/srep31378 · Cited by 65 · see citing works
Pirolli, P. L. T., Mohan, S., Venkatakrishnan, A., Nelson, L., Silva, M., & Springer, A. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8217 · Cited by 68 · see citing works
The Value of Foresight: How Prospection Affects Decision-Making
Pezzulo, G., & Rigoli, F. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00079 · Cited by 66 · see citing works
Action Change Theory: A Reinforcement Learning Perspective on Behavior Change
Vlaev, I., & Dolan, P. (2015) · Review of General Psychology · doi:10.1037/gpr0000029 · Cited by 55 · see citing works
Revisiting the relationship between the P3b and working memory updating
Rac-Lubashevsky, R., & Kessler, Y. (2019) · Biological Psychology · doi:10.1016/j.biopsycho.2019.107769 · Cited by 56 · see citing works
Brain pathways for cognitive-emotional decision making in the human animal
Levine, D. S. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.003 · Cited by 45 · see citing works
Prefrontal Goal Codes Emerge as Latent States in Probabilistic Value Learning
Stoianov, I., Genovesio, A., & Pezzulo, G. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00886 · Cited by 44 · see citing works
The effect of novelty on reinforcement learning
Houillon, A., Lorenz, R. C., Böhmer, W., Rapp, M. A., Heinz, A., Gallinat, J., & Obermayer, K. (2013) · Progress in brain research · doi:10.1016/b978-0-444-62604-2.00021-6 · Cited by 41 · see citing works
Valuation of uncertain and delayed rewards in primate prefrontal cortex
Kim, S., Hwang, J., Seo, H., & Lee, D. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.010 · Cited by 37 · see citing works
Fidgeting as self-evidencing: A predictive processing account of non-goal-directed action
Perrykkad, K., & Hohwy, J. (2019) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2019.100750 · Cited by 37 · see citing works
Goal-Directed and Habitual Control in Human Substance Use: State of the Art and Future Directions
Doñamayor, N., Ebrahimi, C., Arndt, V. A., Weiß, F., Schlagenhauf, F., & Endraß, T. (2022) · Neuropsychobiology · doi:10.1159/000527663 · Cited by 33 · see citing works
The Error-Related Negativity Predicts Self-Control Failures in Daily Life
Overmeyer, R., Berghäuser, J., Dieterich, R., Wolff, M., Goschke, T., & Endraß, T. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2020.614979 · Cited by 32 · see citing works
Goal-Driven Cognition in the Brain: A Computational FrameworkPreprint
O’Reilly, R. C., Hazy, T. E., Mollick, J., Mackie, P., & Herd, S. A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.7591 · Cited by 45 · see citing works
Whoa, No-Go: Evidence consistent with model-based strategy use in horses during an inhibitory task
Louise, E., Cameron‐Whytock, H. A., & Ijichi, C. L. (2024) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2024.106339 · Cited by 29 · see citing works
Shizgal, P. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00020 · Cited by 28 · see citing works
Proactive Action Preparation: Seeing Action Preparation as a Continuous and Proactive Process
Pezzulo, G., & Ognibene, D. (2012) · Motor Control · doi:10.1123/mcj.16.3.386 · Cited by 27 · see citing works
Rationalizable Irrationalities of Choice
Dayan, P. (2014) · Topics in Cognitive Science · doi:10.1111/tops.12082 · Cited by 27 · see citing works
Self-citationLearning and forgetting using reinforced Bayesian change detection
Moens, V., & Zénon, A. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006713 · Cited by 23 · see citing works
Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder
Doñamayor, N., Ebrahimi, C., Garbusow, M., Wedemeyer, F., Schlagenhauf, F., & Heinz, A. (2020) · Current Addiction Reports · doi:10.1007/s40429-020-00333-9 · Cited by 22 · see citing works
Environmental statistics and the trade-off between model-based and TD learning in humans
Simon, D. A., & Daw, N. D. (2011) · neural information processing systems · Cited by 23 · see citing works
Pirolli, P. L. T., Youngblood, G. M., Du, H., Konrad, A., Nelson, L., & Springer, A. (2018) · Human-Computer Interaction · doi:10.1080/07370024.2018.1512414 · Cited by 15 · see citing works
Overmeyer, R., Fürtjes, S., Ersche, K. D., Ehrlich, S., & Endraß, T. (2020) · Personality and Individual Differences · doi:10.1016/j.paid.2020.110029 · Cited by 15 · see citing works
Habit outweighs planning in grasp selection for object manipulation
Herbort, O., Mathew, H., & Kunde, W. (2016) · Cognitive Psychology · doi:10.1016/j.cogpsych.2016.11.008 · Cited by 15 · see citing works
Hierarchical control using networks trained with higher-level forward models
Wayne, G., & Abbott, L. F. (2014) · Neural Computation · doi:10.1162/neco_a_00639 · Cited by 14 · see citing works
Richmond, P., Buesing, L., Giugliano, M., & Vasilaki, E. (2011) · PLoS ONE · doi:10.1371/journal.pone.0018539 · Cited by 13 · see citing works
Persistence is futile: Chasing of past performance in repeated investment choices.
Weiss‐Cohen, L., Newall, P. W. S., & Ayton, P. (2021) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000358 · Cited by 10 · see citing works
Model-free conventions in multi-agent reinforcement learning with heterogeneous preferencesPreprint
Koster, R., McKee, K. R., Everett, R., Weidinger, L., Isaac, W. S., Hughes, E., Duéñez‐Guzmán, E. A., Graepel, T., Botvinick, M., & Leibo, J. Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09054 · Cited by 11 · see citing works
Learned helplessness: expanding on a goal-directed perspective
Duda, J. M., & Joormann, J. (2022) · Cognition & Emotion · doi:10.1080/02699931.2022.2118238 · Cited by 10 · see citing works
Person-by-person prediction of intuitive economic choice
Mengov, G. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.09.002 · Cited by 10 · see citing works
Computational Neural Mechanisms of Goal-Directed Planning and Problem Solving
Fine, J. M., Zarr, N., & Brown, J. W. (2020) · Computational Brain & Behavior · doi:10.1007/s42113-020-00095-7 · Cited by 9 · see citing works
Habitual responses are retrieved from memory, not triggered reflexivelyPreprint
Du, Y., & Haith, A. M. (2023) · doi:10.31234/osf.io/gncsf · Cited by 10 · see citing works
Alizamini, M. M., Chen, X., Nie, L., Dai, Z., Liu, Y., Haghparast, A., Wu, N., Zhang, J., Shen, F., Sui, N., et al. (2022) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2022.110523 · Cited by 9 · see citing works
Pirolli, P. L. T. (2016) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2016.7590670 · Cited by 9 · see citing works
Kavroulakis, E., van Kemenade, B. M., Arikan, B. E., Kircher, T. T. J., & Straube, B. (2022) · Human Brain Mapping · doi:10.1002/hbm.26053 · Cited by 8 · see citing works
Hierarchical principles of embodied reinforcement learning: A reviewPreprint
Eppe, M., Gumbsch, C., Kerzel, M., Nguyen, P. D. H., Butz, M. V., & Wermter, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10147 · Cited by 9 · see citing works
Pluck, G. C., Cerone, A., & Villagomez-Pacheco, D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26236-4_27 · Cited by 7 · see citing works
Circuit mechanisms of navigation strategy learning in mice
Parrini, M., Tricot, G., Caroni, P., & Spolidoro, M. (2023) · Current Biology · doi:10.1016/j.cub.2023.11.047 · Cited by 7 · see citing works
The Effect of Context on Choice and Value
Martino, B. D. (2011) · Elsevier eBooks · doi:10.1016/b978-0-12-381431-9.00005-x · Cited by 8 · see citing works
Models and Methods for Reinforcement Learning
Dayan, P., & Nakahara, H. (2018) · Wiley eBooks · doi:10.1002/9781119170174.epcn513 · Cited by 6 · see citing works
Self-citationGamification Framework for Reinforcement Learning-based Neuropsychology Experiments
Chetitah, M., Müller, J., Deserno, L., Waltmann, M., & von Mammen, S. (2023) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3582437.3587190 · Cited by 5 · see citing works
Nagel, J. (2024) · doi:10.1093/9780191976766.003.0007 · Cited by 4 · see citing works
Electrophysiology of goal-directed versus habitual control during outcome devaluation
Yousuf, M., Heldmann, M., Münte, T. F., & Doñamayor, N. (2019) · Cortex · doi:10.1016/j.cortex.2019.08.002 · Cited by 4 · see citing works
Development and Validation of a Daily Habit Scale
Georgiev, D., Christie, R., Torkamani, M., Song, R., Limousin, P. D., & Jahanshahi, M. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.880023 · Cited by 4 · see citing works
Thalamic regulation of reinforcement learning strategies across prefrontal-striatal networks
Wang, B., Wang, M. B., Lam, N. H., Liu, M., Li, S., Wimmer, R. D., Paz‐Alonso, P. M., Halassa, M. M., & Pleger, B. (2025) · Nature Communications · doi:10.1038/s41467-025-63995-x · Cited by 4 · see citing works
Park, H., Doh, H., Lee, E., Park, H., & Ahn, W. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011692 · Cited by 3 · see citing works
Applications of the Free Energy Principle to Machine Learning and NeurosciencePreprint
Millidge, B. (2021) · arXiv (Cornell University) · doi:10.7488/era/1501 · Cited by 7 · see citing works
Prosocial learning: Model-based or model-free?
Navidi, P., Saeedpour, S., Ershadmanesh, S., Hossein, M. M., & Bahrami, B. (2023) · PLoS ONE · doi:10.1371/journal.pone.0287563 · Cited by 2 · see citing works
Perceptual interventions ameliorate statistical discrimination in learning agents
Duéñez‐Guzmán, E. A., Comanescu, R., Mao, Y., McKee, K. R., Coppin, B., Sadedin, S., Chiappa, S., Vezhnevets, A. S., Bakker, M. A., Bachrach, Y., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2319933121 · Cited by 2 · see citing works
Can we tackle climate change by behavioral hacking of the dopaminergic system?
Munuera, J., & Burguière, E. (2022) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2022.996955 · Cited by 2 · see citing works
Kavroulakis, E., van Kemenade, B. M., Arikan, B. E., Kircher, T., & Straube, B. (2021) · bioRxiv · doi:10.1101/2021.06.19.449116 · Cited by 1 · see citing works
Evaluative Foraging: From Value to ActionPreprint
Clemente, A., & Penacchio, O. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.05.04.652128 · Cited by 2 · see citing works
Learning Nursery Rhymes Using Adaptive Parameter Neurodynamic Programming
Walker, J., & Chalup, S. K. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-14803-8_16 · Cited by 1 · see citing works
In Search of Lost Time A computational model of the representations and dynamics of episodic memory
Wu, Y. X. (2012) · Cited by 1 · see citing works
Modelling Grid Navigation Using Reinforcement Learning Linear Ballistic Accumulators
Venugopal, G., & Bapi, R. S. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10192038 · Cited by 1 · see citing works
Doya, K. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-15754-7.00088-2 · Cited by 1 · see citing works
Intrinsically Motivated Decision Making for Situated, Goal-Driven Agents
Oubbati, M., Fischer, C., & Palm, G. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-08864-8_16 · Cited by 1 · see citing works
Wayne, G. (2013) · doi:10.7916/d8fx7hvf · Cited by 1 · see citing works
Echo State Networks for Hierarchical Cognitive Control
Tukker, R., van Rossum, A., Haselager, P., & Frank, S. L. (2012) · Cited by 1 · see citing works
Overmeyer, R., Fürtjes, S., Ersche, K. D., Ehrlich, S., & Endraß, T. (2019) · doi:10.31234/osf.io/nbwuj · Cited by 1 · see citing works
Statistical discrimination in learning agentsPreprint
Duéñez‐Guzmán, E. A., McKee, K. R., Mao, Y., Coppin, B., Chiappa, S., Vezhnevets, A. S., Bakker, M. A., Bachrach, Y., Sadedin, S., Isaac, W. S., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.11404 · Cited by 6 · see citing works
Complying with norms. a neurocomputational exploration
Colombo, M. (2012) · ERA
Mengov, G. (2015) · Intelligent systems reference library · doi:10.1007/978-3-662-47122-7_3
Vicarious learning: model-based or model-free?Preprint
Navidi, P., Saeedpour, S., Ershadmanesh, S., Hossein, M. M., & Bahrami, B. (2023) · doi:10.31234/osf.io/c3k64
(2023) · Cambridge University Press eBooks · doi:10.1017/9781108755610.004
Doya, K. (2023) · Cambridge University Press eBooks · doi:10.1017/9781108755610.013 · Cited by 1 · see citing works
Larsson, A., Möller, S., Andrekson, S., Nordin, G., Björkstrand, J., & Cervin, M. (2025) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-025-01304-2
A closed loop framework of decision-making and learning in primate prefrontal circuits.
Nallapu, B. T. (2019) · doi:10.70675/ec68f782z2b3ez44f3zb24fzeafa25ff668c · Cited by 1 · see citing works
Activity of dopaminergic neurons during skilled reaching in head-fixed rats
Besserer, C. (2023) · theses.fr (ABES) · doi:10.70675/89acd7a2z37aaz434fzb34dz3b04859d2fa0
- Authors
- Dayan, P.
- Affiliations
- University College London (GB)
- Corresponding author
- Dayan, Peter
- ORCID
- Dayan, Peter
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2009-04
- Volume
- 22
- Issue
- 3
- Pages
- 213-219
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W1967627760
- Cited by
- 93
- References
- 131
- Citations by year
- 2025: 5 · 2024: 2 · 2023: 11 · 2022: 7 · 2021: 7 · 2020: 8 · 2019: 6 · 2018: 4 · 2017: 1 · 2016: 3 · 2015: 9 · 2014: 5 · 2013: 7 · 2012: 7
- FWCI
- 1.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- goal-directed control, operant conditioning, reinforcement learning, cognitive architecture, procedural control, prior distributions, neural correlates, rationality
- MeSH terms
- Animals, Basal Ganglia, Brain, Cognition, Goals, Habituation, Psychophysiologic, Humans, Learning, Models, Neurological, Psychomotor Performance, Reinforcement, Psychology, Prefrontal Cortex
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
