Deep reinforcement learning and its neuroscientific implications

ImageNet: A large-scale hierarchical image database

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works

Human-level control through deep reinforcement learning

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works

Learning Internal Representations by Error Propagation

Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,501 · see citing works

Mastering the game of Go with deep neural networks and tree search

Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works

Playing Atari with Deep Reinforcement Learning

Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · see citing works

Mastering the game of Go without human knowledge

Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · see citing works

Self-Organization and Associative Memory

Kohonen, T. (1989) · Springer Series in Information Sciences · doi:10.1007/978-3-642-88163-3 · Cited by 8,740 · see citing works

Overcoming catastrophic forgetting in neural networks

Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · see citing works

Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position

Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759 · see citing works

Emergence of simple-cell receptive field properties by learning a sparse code for natural images

Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · see citing works

Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects

Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 1× · see citing works

The Organization of Behavior; A Neuropsychological Theory.

Attneave, F., B., M., & Hebb, D. O. (1950) · The American Journal of Psychology · doi:10.2307/1418888 · Cited by 5,146 · see citing works

Receptive fields of single neurones in the cat's striate cortex

Hubel, D. H., & Wiesel, T. N. (1959) · The Journal of Physiology · doi:10.1113/jphysiol.1959.sp006308 · Cited by 4,653 · see citing works

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562 · see citing works

Grandmaster level in StarCraft II using multi-agent reinforcement learning

Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455 · see citing works

Prioritized Experience Replay

Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120 · see citing works

Reactivation of hippocampal ensemble memories during sleep

Wilson, M. A., & McNaughton, B. L. (1994) · Science · doi:10.1126/science.8036517 · Cited by 3,118 · see citing works

Beyond regression : new fools for prediction and analysis in the behavioral sciences

Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,746 · see citing works

Building machines that learn and think like people

Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550 · see citing works

Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works

Performance-optimized hierarchical models predict neural responses in higher visual cortex

Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924 · see citing works

Curiosity-Driven Exploration by Self-Supervised Prediction

Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · see citing works

Context-dependent computation by recurrent dynamics in prefrontal cortex

Mante, V., Sussillo, D., Shenoy, K. V., & Newsome, W. T. (2013) · Nature · doi:10.1038/nature12742 · Cited by 1,807 · see citing works

Model-based influences on humans’ choices and striatal prediction errors

Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works

Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action

Balleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775 · see citing works

Using goal-driven deep learning models to understand sensory cortex

Yamins, D. L. K., & DiCarlo, J. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4244 · Cited by 1,621 · see citing works

Neurons in the orbitofrontal cortex encode economic value

Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · see citing works

Neuroscience-Inspired Artificial Intelligence

Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.011 · Cited by 1,557 · see citing works

Goal-directed instrumental action: contingency and incentive learning and their cortical substrates

Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · see citing works

Optimizing the success of random searches

Viswanathan, G. M., Buldyrev, S. V., Havlin, S., da Luz, M. G. E., Raposo, E. P., & Stanley, E. (1999) · Nature · doi:10.1038/44831 · Cited by 1,525 · see citing works

Hybrid computing using a neural network with dynamic external memory

Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016) · Nature · doi:10.1038/nature20101 · Cited by 1,412 · see citing works

Mastering Atari, Go, chess and shogi by planning with a learned model

Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · RePEc: Research Papers in Economics · Cited by 1,310 · see citing works

Hippocampal place-cell sequences depict future paths to remembered goals

Pfeiffer, B. E., & Foster, D. J. (2013) · Nature · doi:10.1038/nature12112 · Cited by 1,216 · see citing works

A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons

Zipser, D., & Andersen, R. A. (1988) · Nature · doi:10.1038/331679a0 · Cited by 1,172 · see citing works

States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning

Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166 · see citing works

Intrinsic Motivation Systems for Autonomous Mental Development

Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126 · see citing works

Categorical Representation of Visual Stimuli in the Primate Prefrontal Cortex

Freedman, D. J., Riesenhuber, M., Poggio, T., & Miller, E. K. (2001) · Science · doi:10.1126/science.291.5502.312 · Cited by 1,123 · see citing works

Deep neural networks: a new framework for modeling biological vision and brain information processing

Kriegeskorte, N. (2015) · Annual Review of Vision Science · doi:10.1146/annurev-vision-082114-035447 · Cited by 1,093 · see citing works

Toward a rational and mechanistic account of mental effort

Shenhav, A., Musslick, S., Lieder, F., Kool, W., Griffiths, T. L., Cohen, J. D., & Botvinick, M. M. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031526 · Cited by 1,077 · see citing works

Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia

O’Reilly, R. C., & Frank, M. J. (2005) · Neural Computation · doi:10.1162/089976606775093909 · Cited by 1,066 · see citing works

Dota 2 with Large Scale Deep Reinforcement Learning

OpenAI, :, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.06680 · Cited by 1,065 · see citing works

The Hippocampus, Memory, and Place Cells

Eichenbaum, H., Dudchenko, P., Wood, E., Shapiro, M., & Tanila, H. (1999) · Neuron · doi:10.1016/s0896-6273(00)80773-4 · Cited by 1,064 · see citing works

What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior

Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.002 · Cited by 1,055 · see citing works

A deep learning framework for neuroscience

Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027 · see citing works

The hippocampus as a predictive map

Stachenfeld, K. L., Botvinick, M. M., & Gershman, S. J. (2017) · Nature Neuroscience · doi:10.1038/nn.4650 · Cited by 1,000 · see citing works

Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes

Badre, D. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.004 · Cited by 980 · see citing works

Goals and Habits in the Brain

Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977 · see citing works

Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis

Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955 · see citing works

Organizing conceptual knowledge in humans with a gridlike code

Constantinescu, A. O., O’Reilly, J. X., & Behrens, T. E. J. (2016) · Science · doi:10.1126/science.aaf0941 · Cited by 937 · see citing works

Backpropagation and the brain

Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J., & Hinton, G. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0277-3 · Cited by 908 · see citing works

Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning

Nagabandi, A., Kahn, G., Fearing, R. S., & Levine, S. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463189 · Cited by 898 · see citing works

Orbitofrontal Cortex as a Cognitive Map of Task Space

Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.005 · Cited by 885 · see citing works

Feature visualization

Olah, C., Mordvintsev, A., & Schubert, L. (2017) · Distill · doi:10.23915/distill.00007 · Cited by 880 · see citing works

A hierarchy of temporal receptive windows in human cortex

Hasson, U., Yang, E., Vallines, I., Heeger, D. J., & Rubin, N. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.5487-07.2008 · Cited by 874 · see citing works

Reward, Motivation, and Reinforcement Learning

Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855 · see citing works

Object-based attention in the primary visual cortex of the macaque monkey

Roelfsema, P. R., Lamme, V. A. F., & Spekreijse, H. (1998) · Nature · doi:10.1038/26475 · Cited by 814 · see citing works

TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play

Tesauro, G. J. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.215 · Cited by 803 · see citing works

What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated

Kumaran, D., Hassabis, D., & McClelland, J. L. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.05.004 · Cited by 736 · see citing works

Curious model-building control systems

Schmidhuber, J. (1991) · [Proceedings] 1991 IEEE International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.170605 · Cited by 733 · see citing works

Hippocampal replay is not a simple function of experience

Gupta, A. S., van der Meer, M. A. A., Touretzky, D. S., & Redish, A. D. (2010) · Neuron · doi:10.1016/j.neuron.2010.01.034 · Cited by 733 · see citing works

Reinforcement learning in the brain

Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 724 · see citing works

Vector-based navigation using grid-like representations in artificial agents

Banino, A., Barry, C., Uría, B., Blundell, C., Lillicrap, T., Mirowski, P., Pritzel, A., Chadwick, M. J., Degris, T., Modayil, J., et al. (2018) · Nature · doi:10.1038/s41586-018-0102-6 · Cited by 682 · see citing works

Inferring single-trial neural population dynamics using sequential auto-encoders

Pandarinath, C., O’Shea, D. J., Collins, J., Józefowicz, R., Stavisky, S. D., Kao, J. C., Trautmann, E. M., Kaufman, M. T., Ryu, S. I., Hochberg, L. R., et al. (2018) · Nature Methods · doi:10.1038/s41592-018-0109-9 · Cited by 672 · see citing works

Reinforcement learning, fast and slow

Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667 · see citing works

Human-level performance in 3D multiplayer games with population-based reinforcement learning

Jaderberg, M., Czarnecki, W. M., Dunning, I., Marris, L., Lever, G., Castañeda, A. G., Beattie, C., Rabinowitz, N. C., Morcos, A. S., Ruderman, A., et al. (2019) · Science · doi:10.1126/science.aau6249 · Cited by 662 · see citing works

The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal FormationPreprint

Whittington, J. C. R., Muller, T., Mark, S., Chen, G.-F., Barry, C., Burgess, N., & Behrens, T. (2019) · bioRxiv · doi:10.1101/770495 · Cited by 647 · see citing works

Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective

Botvinick, M. M., Niv, Y., & Barto, A. G. (2009) · Cognition · doi:10.1016/j.cognition.2008.08.011 · Cited by 646 · see citing works

Neural computations underlying arbitration between model-based and model-free learning

Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642 · see citing works

Intrinsically Motivated Reinforcement Learning

Singh, S. P., Barto, A. G., & Chentanez, N. (2005) · doi:10.21236/ada440280 · Cited by 614 · see citing works

FeUdal Networks for Hierarchical Reinforcement LearningPreprint

Vezhnevets, A. S., Osindero, S., Schaul, T., Heess, N., Jaderberg, M., Silver, D., & Kavukcuoglu, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01161 · Cited by 608 · see citing works

Solving Rubik's Cube with a Robot Hand

OpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.07113 · Cited by 602 · see citing works

A neural network that finds a naturalistic solution for the production of muscle activity

Sussillo, D., Churchland, M. M., Kaufman, M. T., & Shenoy, K. V. (2015) · Nature Neuroscience · doi:10.1038/nn.4042 · Cited by 598 · see citing works

Prefrontal cortex as a meta-reinforcement learning system

Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596 · see citing works

Humans use directed and random exploration to solve the explore–exploit dilemma.

Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0038199 · Cited by 580 · see citing works

Learning to reinforcement learn

Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579 · see citing works

Anatomy of a decision: Striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal.

Frank, M. J., & Claus, E. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.300 · Cited by 578 · see citing works

Neural representations of events arise from temporal community structure

Schapiro, A. C., Rogers, T. T., Cordova, N. I., Turk-Browne, N. B., & Botvinick, M. M. (2013) · Nature Neuroscience · doi:10.1038/nn.3331 · Cited by 556 · see citing works

A critique of pure learning and what artificial neural networks can learn from animal brains

Zador, A. M. (2019) · Nature Communications · doi:10.1038/s41467-019-11786-6 · Cited by 553 · see citing works

A Task-Optimized Neural Network Replicates Human Auditory Behavior, Predicts Brain Responses, and Reveals a Cortical Processing Hierarchy

Kell, A. J. E., Yamins, D. L. K., Shook, E. N., Norman-Haignere, S. V., & McDermott, J. H. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.044 · Cited by 544 · see citing works

Neural Basis of Reinforcement Learning and Decision Making

Lee, D., Seo, H., & Jung, M. W. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062111-150512 · Cited by 542 · see citing works

What the orbitofrontal cortex does not do

Stalnaker, T. A., Cooch, N. K., & Schoenbaum, G. (2015) · Nature Neuroscience · doi:10.1038/nn.3982 · Cited by 531 · see citing works

Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions

Wimmer, G. E., & Daphna Shohamy, A. (2012) · Science · doi:10.1126/science.1223252 · Cited by 497 · see citing works

Deep Neural Networks as Scientific Models

Cichy, R. M., & KAISER, D. H. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.01.009 · Cited by 453 · see citing works

Reinforcement learning and episodic memory in humans and animals: an integrative framework

Gershman, S. J., & Daw, N. D. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033625 · Cited by 446 · see citing works

A distributional code for value in dopamine-based reinforcement learning

Dabney, W., Kurth-Nelson, Z. L., Uchida, N., Starkweather, C. K., Hassabis, D., Munos, R., & Botvinick, M. (2020) · Nature · doi:10.1038/s41586-019-1924-6 · Cited by 443 · see citing works

Neural Circuitry of Reward Prediction Error

Watabe-Uchida, M., Eshel, N., & Uchida, N. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031109 · Cited by 437 · see citing works

Theories of Error Back-Propagation in the Brain

Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420 · see citing works

How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis

Collins, A. G. E., & Frank, M. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07980.x · Cited by 419 · see citing works

Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective

Singh, S. P., Lewis, R. L., Barto, A. G., & Sorg, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051031 · Cited by 418 · see citing works

Prioritized memory access explains planning and hippocampal replay

Mattar, M. G., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0232-z · Cited by 393 · see citing works

Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks

Hasson, U., Nastase, S. A., & Goldstein, A. (2020) · Neuron · doi:10.1016/j.neuron.2019.12.002 · Cited by 387 · see citing works

Context, learning, and extinction

Gershman, S. J., Blei, D. M., & Niv, Y. (2010) · Psychological Review · doi:10.1037/a0017808 · Cited by 386 · cited 1× · see citing works

Recurrent network models of sequence generation and memory

Rajan, K., Harvey, C. D., & Tank, D. W. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.009 · Cited by 351 · see citing works

Successor Features for Transfer in Reinforcement LearningPreprint

Barreto, A. D. M. S., Dabney, W., Munos, R., Hunt, J. J., Schaul, T., Silver, D., & van Hasselt, H. P. (2016) · arXiv (Cornell University) · doi:10.13140/rg.2.2.35439.18085 · Cited by 349 · see citing works

The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers

Botvinick, M., & Cohen, J. D. (2014) · Cognitive Science · doi:10.1111/cogs.12126 · Cited by 328 · see citing works

Proto-value Functions: A Laplacian Framework for Learning Representation and Control in Markov Decision Processes

Mahadevan, S., & Maggioni, M. (2007) · doi:10.5555/1314498.1314570 · Cited by 287 · see citing works

Value Function Approximation in Reinforcement Learning Using the Fourier Basis

Konidaris, G., Osentoski, S., & Thomas, P. S. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v25i1.7903 · Cited by 277 · see citing works

Hierarchical motor control in mammals and machines

Merel, J., Botvinick, M., & Wayne, G. (2019) · Nature Communications · doi:10.1038/s41467-019-13239-6 · Cited by 274 · see citing works

Multiple gates on working memory

Chatham, C. H., & Badre, D. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2014.08.001 · Cited by 268 · see citing works

Deconstructing the human algorithms for exploration

Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267 · see citing works

A Study on Overfitting in Deep Reinforcement LearningPreprint

Zhang, C., Vinyals, O., Munos, R., & Samy Bengio, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06893 · Cited by 246 · see citing works

An object-oriented representation for efficient reinforcement learning

Diuk, C., Cohen, A., & Littman, M. L. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390187 · Cited by 226 · see citing works

The Option-Critic Architecture

Bacon, P., Harb, J., & Precup, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10916 · Cited by 219 · see citing works

Exploration, novelty, surprise, and free energy minimization

Schwartenbeck, P., FitzGerald, T. H. B., Dolan, R. J., & Friston, K. J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00710 · Cited by 204 · see citing works

Adaptive integration of habits into depth-limited planning defines a habitual-goal–directed spectrum

Keramati, M., Smittenaar, P., Dolan, R. J., & Dayan, P. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1609094113 · Cited by 203 · see citing works

Stabilizing Transformers for Reinforcement Learning

Parisotto, E., Song, H. F., Rae, J. W., Pascanu, R., Gulcehre, C., Jayakumar, S. M., Jaderberg, M., Kaufman, R. L., Clark, A., Noury, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.06764 · Cited by 199 · see citing works

A Pallidus-Habenula-Dopamine Pathway Signals Inferred Stimulus Values

Bromberg-Martin, E. S., Matsumoto, M., Hong, S., & Hikosaka, O. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00158.2010 · Cited by 182 · see citing works

Never Give Up: Learning Directed Exploration Strategies

Badia, A. P., Sprechmann, P., Vitvitskyi, A., Guo, D., Piot, B., Kapturowski, S., Tieleman, O., Arjovsky, M., Pritzel, A., Bolt, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2002.06038 · Cited by 170 · see citing works

Reward-based training of recurrent neural networks for cognitive and value-based tasks

Song, H., Yang, G. R., & Wang, X. (2017) · eLife · doi:10.7554/elife.21492 · Cited by 169 · see citing works

Habitual control of goal selection in humans

Cushman, F. A., & Morris, A. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1506367112 · Cited by 165 · see citing works

Model-Free Episodic Control

Blundell, C., Uria, B., Pritzel, A., Li, Y., Ruderman, A., Leibo, J. Z., Rae, J., Wierstra, D., & Hassabis, D. (2016) · arXiv · doi:10.48550/arxiv.1606.04460 · Cited by 162 · see citing works

Reinstated episodic context guides sampling-based decisions for reward

Bornstein, A. M., & Norman, K. A. (2017) · Nature Neuroscience · doi:10.1038/nn.4573 · Cited by 161 · see citing works

Learning structures: Predictive representations, replay, and generalization

Momennejad, I. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.02.017 · Cited by 159 · see citing works

Multi-agent Reinforcement Learning in Sequential Social DilemmasPreprint

Leibo, J. Z., Zambaldi, V., Lanctot, M., Marecki, J., & Graepel, T. (2017) · arXiv (Cornell University) · doi:10.5555/3091125.3091194 · Cited by 151 · see citing works

Programming robots using reinforcement learning and teaching

Lin, L.-J. (1991) · National Conference on Artificial Intelligence · Cited by 145 · see citing works

Deep Successor Reinforcement LearningPreprint

Kulkarni, T. D., Saeedi, A., Gautam, S., & Gershman, S. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02396 · Cited by 145 · see citing works

Unsupervised Predictive Memory in a Goal-Directed AgentPreprint

Wayne, G., Hung, C.-C., Amos, D., Mirza, M., Ahuja, A., Grabska‐Barwińska, A., Rae, J. W., Mirowski, P., Leibo, J. Z., Santoro, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.10760 · Cited by 137 · see citing works

Deep, Narrow Sigmoid Belief Networks Are Universal Approximators

Sutskever, I., & Hinton, G. E. (2008) · Neural Computation · doi:10.1162/neco.2008.12-07-661 · Cited by 131 · see citing works

Subcortical Substrates of Explore-Exploit Decisions in Primates

Costa, V. D., Mitz, A. R., & Averbeck, B. B. (2019) · Neuron · doi:10.1016/j.neuron.2019.05.017 · Cited by 131 · see citing works

Distral: Robust Multitask Reinforcement Learning

Teh, Y. W. (2025) · TIB Data Manager · doi:10.57702/nufubi9l · Cited by 126 · see citing works

Toward an integration of deep learning and neuroscience

Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119 · see citing works

Modeling the Dynamics of Human Brain Activity with Recurrent Neural Networks

Güçlü, U., & van Gerven, M. A. J. (2017) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2017.00007 · Cited by 113 · see citing works

Recurrent Network Model of the Neural Mechanism of Short-Term Active Memory

Zipser, D. (1991) · Neural Computation · doi:10.1162/neco.1991.3.2.179 · Cited by 108 · see citing works

Reward revaluation biases hippocampal replay content away from the preferred outcome

Carey, A. A., Tanaka, Y., & van der Meer, M. A. A. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0464-6 · Cited by 106 · see citing works

Multiple Systems for Value Learning

Daw, N. D., & O’Doherty, J. P. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00021-8 · Cited by 98 · see citing works

Learning Reward Uncertainty in the Basal Ganglia

Mikhael, J. G., & Bogacz, R. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005062 · Cited by 95 · see citing works

A Laplacian Framework for Option Discovery in Reinforcement LearningPreprint

Machado, M. C., Bellemare, M. G., & Bowling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00956 · Cited by 93 · see citing works

Bayesian Action Decoder for Deep Multi-Agent Reinforcement LearningPreprint

Foerster, J., Song, H., Hughes, E., BURCH, N. R., Dunning, I., Whiteson, S., Botvinick, M., & Bowling, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01458 · Cited by 84 · see citing works

Hierarchical visuomotor control of humanoidsPreprint

Merel, J., Ahuja, A., Pham, V., Tunyasuvunakool, S., Liu, S., Tirumala, D., Heess, N., & Wayne, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09656 · Cited by 82 · see citing works

Action and learning shape the activity of neuronal circuits in the visual cortex

Pakan, J. M. P., Francioni, V., & Rochefort, N. L. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2018.04.020 · Cited by 75 · see citing works

Optimizing agent behavior over long time scales by transporting value

Hung, C.-C., Lillicrap, T., Abramson, J., Wu, Y., Mirza, M., Carnevale, F., Ahuja, A., & Wayne, G. (2019) · Nature Communications · doi:10.1038/s41467-019-13073-w · Cited by 75 · see citing works

Causal Reasoning from Meta-reinforcement Learning

Dasgupta, I., Wang, J., Chiappa, S., Mitrovic, J., Ortega, P., Raposo, D., Hughes, E., Battaglia, P., Botvinick, M., & Kurth-Nelson, Z. (2019) · arXiv · doi:10.48550/arxiv.1901.08162 · Cited by 71 · see citing works

Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits

(2020) · doi:10.1101/2020.03.30.015511 · Cited by 71 · see citing works

On the necessity of abstraction

Konidaris, G. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.11.005 · Cited by 70 · see citing works

COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven ExplorationPreprint

Watters, N., Matthey, L., Bošnjak, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09275 · Cited by 65 · see citing works

Genetic Programming for Reward Function Search

Niekum, S., Barto, A. G., & Spector, L. C. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051436 · Cited by 58 · see citing works

Learning to Perform Physics Experiments via Deep Reinforcement LearningPreprint

Denil, M., Agrawal, P., Kulkarni, T. D., Erez, T., Battaglia, P., & de Freitas, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01843 · Cited by 58 · see citing works

Relational Forward Models for Multi-Agent LearningPreprint

Tacchetti, A., Song, H., Mediano, P. A. M., Zambaldi, V., Rabinowitz, N. C., Graepel, T., Botvinick, M., & Battaglia, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.11044 · Cited by 53 · see citing works

Reinforcement learning, efficient coding, and the statistics of natural tasks

Botvinick, M., Weinstein, A., Solway, A., & Barto, A. G. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.08.009 · Cited by 50 · see citing works

A Comparative Analysis of Expected and Distributional Reinforcement Learning

Lyle, C., Bellemare, M. G., & Castro, P. S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014504 · Cited by 44 · see citing works

The Option Keyboard: Combining Skills in Reinforcement LearningPreprint

Barreto, A. D. M. S., Borsa, D., Hou, S., Comanici, G., Aygün, E., Hamel, P., Toyama, D., Hunt, J. J., Mourad, S., Silver, D., et al. (2019) · arXiv (Cornell University) · doi:10.13140/rg.2.2.12370.50882 · Cited by 42 · see citing works

An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents

Such, F. P., Madhavan, V., Liu, R., Wang, R., Castro, P. S., Li, Y., Zhi, J., Schubert, L., Bellemare, M. G., Clune, J., et al. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/452 · Cited by 34 · see citing works

A neurally plausible model learns successor representations in partially observable environmentsPreprint

Vértes, E., & Sahani, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.09480 · Cited by 28 · see citing works

Exploration in Reinforcement Learning with Deep Covering Options

Jinnai, Y., Park, J. W., Machado, M. C., & Konidaris, G. (2020) · International Conference on Learning Representations · Cited by 28 · see citing works

MEMO: A Deep Network for Flexible Combination of Episodic MemoriesPreprint

Banino, A., Badia, A. P., Koster, R., Chadwick, M. J., Zambaldi, V., Hassabis, D., Barry, C., Botvinick, M., Kumaran, D., & Blundell, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.10913 · Cited by 22 · see citing works

Emergent Systematic Generalization In a Situated AgentPreprint

Hill, F., Lampinen, A. K., Schneider, R. G., Clark, S. C., Botvinick, M., McClelland, J. L., & Santoro, A. (2019) · arXiv (Cornell University) · Cited by 21 · see citing works

When Waiting Is Not an Option: Learning Options With a Deliberation Cost

Harb, J., Bacon, P., Klissarov, M., & Precup, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11831 · Cited by 19 · see citing works

Dopamine Gates Visual Signals in Monkey Prefrontal Cortex Neurons

Stalter, M., Westendorff, S., & Nieder, A. (2020) · Cell Reports · doi:10.1016/j.celrep.2019.11.082 · Cited by 19 · see citing works

The Termination CriticPreprint

Harutyunyan, A., Dabney, W., Borsa, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09996 · Cited by 17 · see citing works

Structure Learning in Motor Control:A Deep Reinforcement Learning ModelPreprint

Weinstein, A., & Botvinick, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.06827 · Cited by 8 · see citing works

The neuroconnectionist research programme

Doerig, A., Sommers, R. P., Seeliger, K., Richards, B., Ismael, J., Lindsay, G. W., Kording, K. P., Konkle, T., van Gerven, M. A. J., Kriegeskorte, N., et al. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00705-w · Cited by 233 · see citing works

Meta-learning in natural and artificial intelligence

Wang, J. X. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.01.002 · Cited by 171 · see citing works

Predictive maps in rats and humans for spatial navigation

de Cothi, W., Nyberg, N., Griesbauer, E.-M., Ghanamé, C., Zisch, F., Lefort, J. M., Fletcher, L., Newton, C., Renaudineau, S., Bendor, D., et al. (2022) · Current Biology · doi:10.1016/j.cub.2022.06.090 · Cited by 109 · see citing works

Emergent behaviour and neural dynamics in artificial agents tracking odour plumes

Singh, S. H., van Breugel, F., Rao, R. P. N., & Brunton, B. W. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00599-w · Cited by 61 · see citing works

Tasks and their role in visual neuroscience

Kay, K., Bonnen, K., Denison, R. N., Arcaro, M. J., & Barack, D. L. (2023) · Neuron · doi:10.1016/j.neuron.2023.03.022 · Cited by 61 · see citing works

Emergence of belief-like representations through reinforcement learning

Hennig, J. A., Romero Pinto, S. A., Yamaguchi, T., Linderman, S. W., Uchida, N., & Gershman, S. J. (2023) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011067 · Cited by 41 · see citing works

Optimization of Neuroprosthetic Vision via End-to-end Deep Reinforcement Learning

Küçükoğlu, B., Rueckauer, B., Ahmad, N., de Ruyter van Steveninck, J., Güçlü, U., & van Gerven, M. (2022) · doi:10.1101/2022.02.25.482017 · Cited by 25 · see citing works

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6 · see citing works

A multi-agent reinforcement learning model of reputation and cooperation in human groups

McKee, K. R., Hughes, E., Zhu, T. O., Chadwick, M. J., Koster, R., Castaneda, A. G., Beattie, C., Graepel, T., Botvinick, M., & Leibo, J. Z. (2021) · arXiv · doi:10.48550/arxiv.2103.04982 · Cited by 5 · see citing works

If deep learning is the answer, then what is the question?Preprint

Saxe, A., Nelli, S., & Summerfield, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.07580 · Cited by 8 · see citing works

If deep learning is the answer, what is the question?

Saxe, A., Nelli, S., & Summerfield, C. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-00395-8 · Cited by 397 · see citing works

How Artificial Intelligence is Supporting Neuroscience Research: A Discussion About Foundations, Methods and Applications

Gonzalez, R. T., Riascos, J. A., & Barone, D. A. C. (2017) · Communications in computer and information science · doi:10.1007/978-3-319-71011-2_6 · Cited by 12 · see citing works

Deep Learning in BioinformaticsPreprint

Min, S., Lee, B., & Yoon, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06430 · Cited by 20 · see citing works

Neuroscientism, the neuroscience of learning

Lim, D. H., Chai, D. S., Park, S., & Doo, M. Y. (2019) · European journal of training and development · doi:10.1108/ejtd-03-2019-0033 · Cited by 20 · see citing works