The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
The ubiquity of model-based reinforcement learning
Open accessAbstract
The reward prediction error (RPE) theory of dopamine (DA) function has enjoyed great success in the neuroscience of learning and decision-making. This theory is derived from model-free reinforcement learning (RL), in which choices are made simply on the basis of previously realized rewards. Recently, attention has turned to correlates of more flexible, albeit computationally complex, model-based methods in the brain. These methods are distinguished from model-free learning by their evaluation of candidate actions using expected future outcomes according to a world model. Puzzlingly, signatures from these computations seem to be pervasive in the very same regions previously thought to support model-free learning. Here, we review recent behavioral and neural evidence about these two systems, in attempt to reconcile their enigmatic cohabitation in the brain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“From an RL perspective, stress shifts the balance toward habits over lengthy planning by increasing experienced time pressure (Doll et al. 2012).”
From Psychology of Habit. (2016) · cites this paper as Doll et al. 2012
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Given the profusion of direct and indirect connections between neural circuits underlying goal-directed and habitual (model-free) behaviors, goals can have a biasing influence on habit formation (Doll et al. 2012).”
From Psychology of Habit. (2016) · cites this paper as Doll et al. 2012
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1978) · UA Campus Repository (The University of Arizona) · Cited by 8,010
Cognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationThe Hippocampus as a Cognitive Map
Ranck, J. B. (1979) · American Journal of Psychiatry · doi:10.1176/ajp.136.10.1353 · Cited by 2,373
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Self-citationBalleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598
Sutton, R. S. (1990) · Elsevier eBooks · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,381
Cited by
Sorted by citations
Wood, W., & Rünger, D. (2016) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033417 · Cited by 1,445
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Model-based Reinforcement Learning: A Survey
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000086 · Cited by 871
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J. R., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 460
Oldham, S., Murawski, C., Fornito, A., Youssef, G. J., Yücel, M., & Lorenzetti, V. (2018) · Human Brain Mapping · doi:10.1002/hbm.24184 · Cited by 449
Mechanisms of motivation–cognition interaction: challenges and opportunities
Braver, T. S., Krug, M. K., Chiew, K. S., Kool, W., Westbrook, J. A., Clement, N. J., Adcock, R. A., Barch, D. M., Botvinick, M., Carver, C. S., et al. (2014) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-014-0300-0 · Cited by 452
Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation
Dayan, P., & Berridge, K. C. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0277-8 · Cited by 357
The basolateral amygdala in reward learning and addiction
Wassum, K. M., & Izquierdo, A. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.08.017 · Cited by 306
Goschke, T. (2014) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1410 · Cited by 293
Reinforcement learning in artificial and biological systems
Neftci, E. O., & Averbeck, B. B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0025-4 · Cited by 268
Aliramezani, M., Koch, C. R., & Shahbakhti, M. (2021) · Progress in Energy and Combustion Science · doi:10.1016/j.pecs.2021.100967 · Cited by 304
Computational approaches to psychiatry
Stephan, K. E., & Mathys, C. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.007 · Cited by 260
The drift diffusion model as the choice rule in reinforcement learning
Pedersen, M. L., Frank, M. J., & Biele, G. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1199-y · Cited by 295
Model-based choices involve prospective neural activity
Doll, B. B., Duncan, K. D., Simon, D. A., Shohamy, D., & Daw, N. D. (2015) · Nature Neuroscience · doi:10.1038/nn.3981 · Cited by 289
Self-citationMiller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280
From Creatures of Habit to Goal-Directed Learners
Decker, J. H., Otto, A. R., Daw, N. D., & Hartley, C. A. (2016) · Psychological Science · doi:10.1177/0956797616639301 · Cited by 281
Self-citationDeserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1417219112 · Cited by 233
Reinforcement learning in depression: a review of computational research
Chen, C., Takahashi, T., Nakagawa, S., Inoue, T., & Kusumi, I. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.05.005 · Cited by 233
Executive control and decision-making in the prefrontal cortex
Domenech, P., & Koechlin, É. (2014) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2014.10.007 · Cited by 261
Acute stress and episodic memory retrieval: neurobiological mechanisms and behavioral consequences
Gagnon, S. A., & Wagner, A. D. (2016) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.12996 · Cited by 255
Barch, D. M., Pagliaccio, D., & Luking, K. (2015) · Current Topics in Behavioral Neurosciences · doi:10.1007/7854_2015_376 · Cited by 218
Towards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224
Shin, J. D., Tang, W., & Jadhav, S. P. (2019) · Neuron · doi:10.1016/j.neuron.2019.09.012 · Cited by 199
Behaviour Centred Design: towards an applied science of behaviour change
Aunger, R. V., & Curtis, V. A. (2016) · Health Psychology Review · doi:10.1080/17437199.2016.1219673 · Cited by 234
VTA neurons coordinate with the hippocampal reactivation of spatial experience
Gomperts, S. N., Kloosterman, F., & Wilson, M. A. (2015) · eLife · doi:10.7554/elife.05360 · Cited by 191
The algorithmic anatomy of model-based evaluation
Daw, N. D., & Dayan, P. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0478 · Cited by 195
Self-citationAttention as an effect not a cause
Krauzlis, R. J., Bollimunta, A., Arcizet, F., & Wang, L. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.05.008 · Cited by 198
Abstract context representations in primate amygdala and prefrontal cortex
Saez, A., Rigotti, M., Ostojic, S., Fusi, S., & Salzman, C. D. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.024 · Cited by 183
Skvortsova, V., Palminteri, S., & Pessiglione, M. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.1350-14.2014 · Cited by 177
Decision Making: From Neuroscience to Psychiatry
Lee, D. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.008 · Cited by 178
Model-based predictions for dopamine
Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.10.006 · Cited by 161
Humans primarily use model-based inference in the two-stage task
Feher da Silva, C., & Hare, T. A. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0905-y · Cited by 153
Model-based hierarchical reinforcement learning and human action control
Botvinick, M., & Weinstein, A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0480 · Cited by 166
The Computational Development of Reinforcement Learning during Adolescence
Palminteri, S., Kilford, E. J., Coricelli, G., & Blakemore, S. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004953 · Cited by 158
Murty, V. P., Calabro, F. J., & Luna, B. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.07.034 · Cited by 167
Instrumental learning of traits versus rewards: dissociable neural correlates and effects on choice
Hackel, L. M., Doll, B. B., & Amodio, D. M. (2015) · Nature Neuroscience · doi:10.1038/nn.4080 · Cited by 148
Self-citationCavanagh, J. F. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.02.007 · Cited by 154
Episodic memories predict adaptive value-based decision-making.
Murty, V. P., FeldmanHall, O., Hunter, L. E., Phelps, E. A., & Davachi, L. (2016) · Journal of Experimental Psychology General · doi:10.1037/xge0000158 · Cited by 149
What Are Memories For? The Hippocampus Bridges Past Experience with Future Decisions
Biderman, N., Bakkour, A., & Shohamy, D. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.04.004 · Cited by 160
Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework
Sadacca, B. F., Jones, J. L., & Schoenbaum, G. M. (2016) · eLife · doi:10.7554/elife.13665 · Cited by 136
Taking stock of value in the orbitofrontal cortex
Knudsen, E. B., & Wallis, J. D. (2022) · Nature reviews. Neuroscience · doi:10.1038/s41583-022-00589-2 · Cited by 148
Of goals and habits: age-related and individual differences in goal-directed decision-making
Eppinger, B., Walter, M., Heekeren, H. R., & Li, S. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00253 · Cited by 141
Moral Learning: Conceptual foundations and normative relevance
Railton, P. (2016) · Cognition · doi:10.1016/j.cognition.2016.08.015 · Cited by 136
Wirth, S., Baraduc, P., Planté, A., Pinède, S., & Duhamel, J.-R. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2001045 · Cited by 129
Dopamine selectively remediates ‘model-based’ reward learning: a computational approach
Sharp, M. E., Foerde, K., Daw, N. D., & Shohamy, D. (2015) · Brain · doi:10.1093/brain/awv347 · Cited by 133
Self-citationBeyond dichotomies in reinforcement learning
Collins, A. G. E., & Cockburn, J. (2020) · Nature reviews. Neuroscience · doi:10.1038/s41583-020-0355-6 · Cited by 147
Reinforcement-learning in fronto-striatal circuits
Averbeck, B. B., & O’Doherty, J. P. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01108-0 · Cited by 132
Lateral orbitofrontal cortex anticipates choices and integrates prior with current information
Nogueira, R., Abolafia, J. M., Drugowitsch, J., Balaguer‐Ballester, E., Sanchez-Vives, M. V., & Moreno‐Bote, R. (2017) · Nature Communications · doi:10.1038/ncomms14823 · Cited by 128
Computational Rationality as a Theory of Interaction
Oulasvirta, A., Jokinen, J., & Howes, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517739 · Cited by 147
Prefrontal Cortex Predicts State Switches during Reversal Learning
Bartolo, R., & Averbeck, B. B. (2020) · Neuron · doi:10.1016/j.neuron.2020.03.024 · Cited by 132
Gaussian Processes for Learning and Control: A Tutorial with Examples
Liu, M., Chowdhary, G. V., da Silva, B. C., Liu, S.-Y., & How, J. P. (2018) · IEEE Control Systems · doi:10.1109/mcs.2018.2851010 · Cited by 128
Habitual versus goal-directed action control in Parkinson disease
de Wit, S., Barker, R. A., Dickinson, A. D., & Cools, R. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21514 · Cited by 116
Learning action-oriented models through active inference
Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007805 · Cited by 121
The role of the orbitofrontal cortex in alcohol use, abuse, and dependence
Moorman, D. E. (2018) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2018.01.010 · Cited by 114
Closed-Loop Theta Stimulation in the Orbitofrontal Cortex Prevents Reward-Based Learning
Knudsen, E. B., & Wallis, J. D. (2020) · Neuron · doi:10.1016/j.neuron.2020.02.003 · Cited by 110
Lewis, M. A., Fagan, W. F., Auger‐Méthé, M., Frair, J. L., Fryxell, J. M., Gros, C., Gurarie, E. D., Healy, S. D., & Merkle, J. A. (2021) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2021.681704 · Cited by 115
Models, movements, and minds: bridging the gap between decision making and action
Wispinski, N. J., Gallivan, J. P., & Chapman, C. S. (2018) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13973 · Cited by 112
Using fMRI to study reward processing in humans: past, present, and future
Wang, K. S., Smith, D. V., & Delgado, M. R. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00333.2015 · Cited by 105
Neural Basis of Strategic Decision Making
Lee, D., & Seo, H. (2015) · Trends in Neurosciences · doi:10.1016/j.tins.2015.11.002 · Cited by 114
Variability in Dopamine Genes Dissociates Model-Based and Model-Free Reinforcement Learning
Doll, B. B., Bath, K. G., Daw, N. D., & Frank, M. J. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1901-15.2016 · Cited by 110
Self-citationSpecies Distribution Modeling for Machine Learning Practitioners: A Review
Beery, S., Cole, E., Parker, J., Perona, P., & Winner, K. (2021) · ACM SIGCAS Conference on Computing and Sustainable Societies · doi:10.1145/3460112.3471966 · Cited by 100
Decoding cognition from spontaneous neural activity
Liu, Y., Nour, M. M., Schuck, N. W., Behrens, T. E. J., & Dolan, R. J. (2022) · Nature reviews. Neuroscience · doi:10.1038/s41583-022-00570-z · Cited by 107
The structure of reinforcement-learning mechanisms in the human brain
O’Doherty, J. P., Lee, S. W., & McNamee, D. (2014) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2014.10.004 · Cited by 106
Howard, J. D., & Kahnt, T. (2018) · Nature Communications · doi:10.1038/s41467-018-04055-5 · Cited by 95
Reflections on dynamics, adaptation and control: A cognitive architecture for mental models
van Ments, L., & Jan Treur, A. (2021) · Cognitive Systems Research · doi:10.1016/j.cogsys.2021.06.004 · Cited by 97
Multiple memory systems as substrates for multiple decision systems
Doll, B. B., Shohamy, D., & Daw, N. D. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.04.014 · Cited by 101
Self-citationExplicit and implicit reinforcement learning across the psychosis spectrum.
Barch, D. M., Carter, C. S., Gold, J. M., Johnson, S. L., Kring, A. M., MacDonald, A. W., Pizzagalli, D. A., Ragland, J. D., Silverstein, S. M., & Strauss, M. E. (2017) · Journal of Abnormal Psychology · doi:10.1037/abn0000259 · Cited by 97
Markou, A., Salamone, J. D., Bussey, T. J., Mar, A. C., Brunner, D., Gilmour, G., & Balsam, P. D. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.08.007 · Cited by 95
Perez, O. D., & Dickinson, A. (2020) · Psychological Review · doi:10.1037/rev0000201 · Cited by 98
Learning to predict or predicting to learn?
Rabagliati, H., Gambi, C., & Pickering, M. J. (2015) · Language Cognition and Neuroscience · doi:10.1080/23273798.2015.1077979 · Cited by 99
Konovalov, A., & Krajbich, I. M. (2016) · Nature Communications · doi:10.1038/ncomms12438 · Cited by 98
Gerraty, R. T., Davidow, J. Y., Wimmer, G. E., Kahn, I., & Shohamy, D. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0185-14.2014 · Cited by 80
Adolescent Development of Value-Guided Goal Pursuit
Davidow, J. Y., Insel, C., & Somerville, L. H. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.05.003 · Cited by 87
Reiter, A. M. F., Deserno, L., Kallert, T. W., Heinze, H., Heinz, A., & Schlagenhauf, F. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.4322-15.2016 · Cited by 85
Huys, Q. J. M., Guitart‐Masip, M., Dolan, R. J., & Dayan, P. (2015) · Clinical Psychological Science · doi:10.1177/2167702614562040 · Cited by 85
Goal-oriented and habitual decisions: Neural signatures of model-based and model-free learning
Huang, Y., Yaple, Z. A., & Yu, R. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116834 · Cited by 80
Devaluation and sequential decisions: linking goal-directed and model-based behavior
Friedel, E., Koch, S., Wendt, J., Heinz, A., Deserno, L., & Schlagenhauf, F. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00587 · Cited by 85
Lucantonio, F., Caprioli, D., & Schoenbaum, G. M. (2013) · Neuropharmacology · doi:10.1016/j.neuropharm.2013.05.033 · Cited by 80
Posttraumatic Stress Disorder and the Developing Adolescent Brain
Cisler, J. M., & Herringa, R. J. (2020) · Biological Psychiatry · doi:10.1016/j.biopsych.2020.06.001 · Cited by 81
Amygdala-cortical collaboration in reward learning and decision making
Wassum, K. M. (2022) · eLife · doi:10.7554/elife.80926 · Cited by 88
Emotions as ComputationsPreprint
Emanuel, A., & Eldar, E. (2022) · PsyArXiv (OSF Preprints) · doi:10.1016/j.neubiorev.2022.104977 · Cited by 84
Neurofeedback through the lens of reinforcement learning
Lubianiker, N., Paret, C., Dayan, P., & Hendler, T. (2022) · Trends in Neurosciences · doi:10.1016/j.tins.2022.03.008 · Cited by 85
Model-based learning: a synthesis of theory and research
Seel, N. M. (2017) · Educational Technology Research and Development · doi:10.1007/s11423-016-9507-9 · Cited by 78
Spatial Representations in the Human Brain
Herweg, N. A., & Kahana, M. J. (2018) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2018.00297 · Cited by 77
Gard, T., Taquet, M., Dixit, R., Hölzel, B. K., Dickerson, B. C., & Lazar, S. W. (2015) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2015.00137 · Cited by 65
Scholl, J., & Klein-Flügge, M. C. (2017) · Behavioural Brain Research · doi:10.1016/j.bbr.2017.09.050 · Cited by 69
Kim, D., Park, G. Y., O’Doherty, J. P., & Lee, S. W. (2019) · Nature Communications · doi:10.1038/s41467-019-13632-1 · Cited by 64
Water optimization technique for precision irrigation system using IoT and machine learning
Vianny, D. M. M., John, A., Mohan, S., Sarlan, A. B., Adimoolam, M., & Ahmadian, A. (2022) · Sustainable Energy Technologies and Assessments · doi:10.1016/j.seta.2022.102307 · Cited by 63
Deserno, L., Wilbertz, T., Reiter, A. M. F., Horstmann, A., Neumann, J., Villringer, A., Heinze, H. –., & Schlagenhauf, F. (2015) · Translational Psychiatry · doi:10.1038/tp.2015.139 · Cited by 66
Issues or Identity? Cognitive Foundations of Voter Choice
Jenke, L., & Huettel, S. A. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.08.013 · Cited by 68
Computational models of drug use and addiction: A review.
Mollick, J., & Hedy Kober, A. (2020) · Journal of Abnormal Psychology · doi:10.1037/abn0000503 · Cited by 64
Explainable reinforcement learning for broad-XAI: a conceptual framework and survey
Dazeley, R., Vamplew, P. W., & Cruz, F. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08423-1 · Cited by 91
Safety out of control: dopamine and defence
Lloyd, K., & Dayan, P. (2016) · Behavioral and Brain Functions · doi:10.1186/s12993-016-0099-7 · Cited by 64
Striatal dopamine, reward, and decision making in schizophrenia
Deserno, L., Schlagenhauf, F., & Heinz, A. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/ldeserno · Cited by 58
A cortical circuit mechanism for structural knowledge-based flexible sensorimotor decision-making
Liu, Y., Xin, Y., & Xu, N.-L. (2021) · Neuron · doi:10.1016/j.neuron.2021.04.014 · Cited by 57
The neural basis of consciousness
Frith, C. (2019) · Psychological Medicine · doi:10.1017/s0033291719002204 · Cited by 60
Modeling other minds: Bayesian inference explains human choices in group decision-making
Khalvati, K., Park, S. A., Mirbagheri, S., Philippe, R., Sestito, M., Dreher, J., & Rao, R. P. N. (2019) · Science Advances · doi:10.1126/sciadv.aax8783 · Cited by 61
Reinforcement learning and its connections with neuroscience and psychology
Subramanian, A., Chitlangia, S., & Baths, V. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.10.003 · Cited by 66
Motivational Processes Underlying Substance Abuse Disorder
Meyer, P., King, C. P., & Ferrario, C. R. (2015) · Current topics in behavioral neurosciences · doi:10.1007/7854_2015_391 · Cited by 59
Sjoerds, Z., Dietrich, A., Deserno, L., de Wit, S., Villringer, A., Heinze, H., Schlagenhauf, F., & Horstmann, A. (2016) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2016.00234 · Cited by 54
Metadata
- Authors
- Doll, B. B., Simon, D. A., & Daw, N. D.
- Affiliations
- Columbia University (US)
- New York University (US)
- Corresponding author
- Daw, Nathaniel D
- Venue
- Current Opinion in Neurobiology
- Publisher
- Elsevier BV
- Published
- 2012-12
- Volume
- 22
- Issue
- 6
- Pages
- 1075-1081
- ISSN
- 0959-4388 (ISSN-L)
- PMC
- PMC3513648
- OpenAlex
- W1977114987
- Cited by
- 380
- References
- 64
- Citations by year
- 2026: 13 · 2025: 26 · 2024: 40 · 2023: 26 · 2022: 35 · 2021: 34 · 2020: 33 · 2019: 29 · 2018: 29 · 2017: 17 · 2016: 32 · 2015: 31 · 2014: 21 · 2013: 13
- FWCI
- 1.65
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Current Opinion in Neurobiology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- McKnight Foundation
- McDonnell Foundation
- NIMH (1R01MH087882-01)
- NINDS (1R01NS078784-01)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- model-based reinforcement learning, model-free reinforcement learning, reward prediction error, dopamine, decision-making, world models, neural correlates of learning, computational psychiatry
- MeSH terms
- Animals, Brain, Choice Behavior, Decision Making, Humans, Models, Neurological, Reinforcement, Psychology, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
