Generalization of value in reinforcement learning by humans
Open accessAbstract
Research in decision‐making has focused on the role of dopamine and its striatal targets in guiding choices via learned stimulus–reward or stimulus–response associations, behavior that is well described by reinforcement learning theories. However, basic reinforcement learning is relatively limited in scope and does not explain how learning about stimulus regularities or relations may guide decision‐making. A candidate mechanism for this type of learning comes from the domain of memory, which has highlighted a role for the hippocampus in learning of stimulus–stimulus relations, typically dissociated from the role of the striatum in stimulus–response learning. Here, we used functional magnetic resonance imaging and computational model‐based analyses to examine the joint contributions of these mechanisms to reinforcement learning. Humans performed a reinforcement learning task with added relational structure, modeled after tasks used to isolate hippocampal contributions to memory. On each trial participants chose one of four options, but the reward probabilities for pairs of options were correlated across trials. This (uninstructed) relationship between pairs of options potentially enabled an observer to learn about option values based on experience with the other options and to generalize across them. We observed blood oxygen level‐dependent (BOLD) activity related to learning in the striatum and also in the hippocampus. By comparing a basic reinforcement learning model to one augmented to allow feedback to generalize between correlated options, we tested whether choice behavior and BOLD activity were influenced by the opportunity to generalize across correlated options. Although such generalization goes beyond standard computational accounts of reinforcement learning and striatal BOLD, both choices and striatal BOLD activity were better explained by the augmented model. Consistent with the hypothesized role for the hippocampus in this generalization, functional connectivity between the ventral striatum and hippocampus was modulated, across participants, by the ability of the augmented model to capture participants’ choice. Our results thus point toward an interactive model in which striatal reinforcement learning systems may employ relational representations typically associated with the hippocampus.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In reanalysis of a monetary reward experiment, ventral striatum tracked appetitive prediction errors to monetary rewards (t(20)=5.77, p<0.001).
“As expected, in contradistinction to the main study, appetitive PEs to monetary rewards were tracked by activity in the VS. (t(20) = 5.77, p < 0.001), but not the PAG (PAG–appetitive: t(20) = 1.54, p = 0.14; see Supplementary Fig. 2).”
From Representation of aversive prediction errors in the human periaqueductal gray (2014) · cites this paper as [23]
In that reanalysis, periaqueductal gray did not track appetitive prediction errors to monetary rewards (t(20)=1.54, p=0.14).
“As expected, in contradistinction to the main study, appetitive PEs to monetary rewards were tracked by activity in the VS. (t(20) = 5.77, p < 0.001), but not the PAG (PAG–appetitive: t(20) = 1.54, p = 0.14; see Supplementary Fig. 2).”
From Representation of aversive prediction errors in the human periaqueductal gray (2014) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 19,308 · see citing works
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., & Joliot, M. (2002) · NeuroImage · doi:10.1006/nimg.2001.0978 · Cited by 16,319 · see citing works
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Cragg, J. G., & Zarembka, P. (1975) · Canadian Journal of Economics/Revue canadienne d économique · doi:10.2307/134075 · Cited by 8,385 · see citing works
Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans.
Squire, L. R. (1992) · Psychological Review · doi:10.1037/0033-295x.99.2.195 · Cited by 5,376 · see citing works
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785 · see citing works
Psychophysiological and Modulatory Interactions in Neuroimaging
Friston, K. J., Buechel, C., Fink, G. R., Morris, J., Rolls, E., & Dolan, R. J. (1997) · NeuroImage · doi:10.1006/nimg.1997.0291 · Cited by 3,097 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
Self-citationAnticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Self-citationA framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · see citing works
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964 · see citing works
Assessing the significance of focal activations using their spatial extent
Friston, K. J., Worsley, K. J., Frackowiak, R. S. J., Mazziotta, J. C., & Evans, A. C. (1994) · Human Brain Mapping · doi:10.1002/hbm.460010306 · Cited by 1,907 · see citing works
Valid conjunction inference with the minimum statistic
Nichols, T. E., Brett, M., Andersson, J., Wager, T. D., & Poline, J. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.005 · Cited by 1,885 · see citing works
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works
Self-citationA neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777 · see citing works
Neural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721 · see citing works
Swanson, L. W. (1982) · Brain Research Bulletin · doi:10.1016/0361-9230(82)90145-9 · Cited by 1,718 · see citing works
Behavioral Theories and the Neurophysiology of Reward
Schultz, W. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.56.091103.070229 · Cited by 1,637 · see citing works
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584 · see citing works
Memory, amnesia, and the hippocampal system
(1994) · Choice Reviews Online · doi:10.5860/choice.31-5148 · Cited by 1,533 · see citing works
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · see citing works
Experience-weighted Attraction Learning in Normal Form Games
Camerer, C. F., & Ho, T. (1999) · Econometrica · doi:10.1111/1468-0262.00054 · Cited by 1,442 · see citing works
Bayesian model selection for group studies
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.03.025 · Cited by 1,423 · see citing works
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190 · see citing works
Item, context and relational episodic encoding in humans
Davachi, L. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.10.012 · Cited by 1,138 · see citing works
Interactive memory systems in the human brain
Poldrack, R. A., Clark, J., Paré-Blagoev, E. J., Shohamy, D., Creso Moyano, J., Myers, C., & Gluck, M. A. (2001) · Nature · doi:10.1038/35107080 · Cited by 1,132 · see citing works
Matching Behavior and the Representation of Value in the Parietal Cortex
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004) · Science · doi:10.1126/science.1094765 · Cited by 1,030 · see citing works
Reward-motivated learning: mesolimbic activation precedes memory formation
Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., & Gabrieli, J. D. E. (2006) · Neuron · doi:10.1016/j.neuron.2006.03.036 · Cited by 977 · see citing works
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · see citing works
Generalisability, Random Effects & Population Inference
Holmes, A. P., & Friston, K. J. (1998) · NeuroImage · doi:10.1016/s1053-8119(18)31587-8 · Cited by 906 · see citing works
Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions
Plassmann, H., O'Doherty, J., & Rangel, A. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2131-07.2007 · Cited by 855 · see citing works
Optimized EPI for fMRI studies of the orbitofrontal cortex
Deichmann, R., Gottfried, J. A., Hutton, C., & Turner, R. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00073-9 · Cited by 821 · see citing works
COGNITIVE NEUROSCIENCE OF HUMAN MEMORY
Gabrieli, J. D. E. (1998) · Annual Review of Psychology · doi:10.1146/annurev.psych.49.1.87 · Cited by 808 · see citing works
Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792 · see citing works
Kelley, A. E., & Domesick, V. B. (1982) · Neuroscience · doi:10.1016/0306-4522(82)90198-1 · Cited by 746 · see citing works
What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?
Doya, K. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00046-5 · Cited by 739 · see citing works
Daphna Shohamy, A., & Adcock, R. A. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.08.002 · Cited by 702 · see citing works
Localization of monoamines in the lower brain stem
Dahlström, A. B., & Fuxé, K. (1964) · Cellular and Molecular Life Sciences · doi:10.1007/bf02147990 · Cited by 680 · see citing works
Neuroeconomics: Decision Making and the Brain
Rushworth, M. F. S., & Walton, M. E. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.005 · Cited by 669 · see citing works
Boorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665 · see citing works
The discovery of structural form
Kemp, C., & Tenenbaum, J. B. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0802631105 · Cited by 656 · see citing works
Nonlinear event‐related responses in fMRI
Friston, K. J., Josephs, O., Rees, G. E., & Turner, R. (1998) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910390109 · Cited by 629 · see citing works
Structure and strength in causal induction☆
Griffiths, T. L., & Tenenbaum, J. B. (2005) · Cognitive Psychology · doi:10.1016/j.cogpsych.2005.05.004 · Cited by 618 · see citing works
The hippocampus and memory for orderly stimulus relations
Dusek, J. A., & Eichenbaum, H. (1997) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.94.13.7109 · Cited by 610 · see citing works
Modulation of competing memory systems by distraction
Foerde, K., Knowlton, B. J., & Poldrack, R. A. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0602659103 · Cited by 600 · see citing works
Huang, Y.-Y., & Kandel, E. R. (1995) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.92.7.2446 · Cited by 544 · see citing works
Integrating Memories in the Human Brain: Hippocampal-Midbrain Encoding of Overlapping Events
Daphna Shohamy, A., & Wagner, A. D. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.023 · Cited by 525 · see citing works
Neural correlates of mentalizing-related computations during strategic interactions in humans
Hampton, A. N., Bossaerts, P. L., & O’Doherty, J. P. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0711099105 · Cited by 518 · see citing works
Model‐Based fMRI and Its Application to Reward Learning and Decision Making
O’Doherty, J. P., Hampton, A. N., & Kim, H. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.022 · Cited by 509 · see citing works
A unified framework for addiction: Vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0800472x · Cited by 501 · see citing works
Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498 · see citing works
Schott, B. H., Minuzzi, L., Krebs, R. M., Elmenhorst, D., Lang, M., Winz, O. H., Seidenbecher, C. I., Coenen, H. H., Heinze, H.-J., Zilles, K., et al. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2058-08.2008 · Cited by 496 · see citing works
Hippocampal mediation of stimulus representation: A computational theory
Gluck, M. A., & Myers, C. E. (1993) · Hippocampus · doi:10.1002/hipo.450030410 · Cited by 496 · see citing works
DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485 · see citing works
Trial-by-trial data analysis using computational models
Daw, N. D. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199600434.003.0001 · Cited by 472 · see citing works
Self-citationThe computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 462 · see citing works
Self-citationBayesian theories of conditioning in a changing world
Courville, A. C., Daw, N. D., & Touretzky, D. S. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.004 · Cited by 457 · see citing works
Self-citationAn automatic valuation system in the human brain: Evidence from functional neuroimaging
Lebreton, M., Jorge, S., Michel, V., Thirion, B., & Pessiglione, M. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.040 · Cited by 455 · see citing works
Activity in human ventral striatum locked to errors of reward prediction
Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002) · Nature Neuroscience · doi:10.1038/nn802 · Cited by 451 · see citing works
Kim, H., Shimojo, S., & O’Doherty, J. P. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040233 · Cited by 447 · see citing works
Dorris, M. C., & Glimcher, P. W. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.009 · Cited by 428 · see citing works
The Role of the Hippocampus in Prediction and Imagination
Buckner, R. L. (2010) · Annual Review of Psychology · doi:10.1146/annurev.psych.60.110707.163508 · Cited by 419 · see citing works
SCHACTER, D. L. (1990) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1990.tb48909.x · Cited by 418 · see citing works
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390 · see citing works
Self-citationOtmakhova, N. A., & Lisman, J. E. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-23-07478.1996 · Cited by 377 · see citing works
The Eyes Have It: Hippocampal Activity Predicts Expression of Memory in Eye Movements
Hannula, D. E., & Ranganath, C. (2009) · Neuron · doi:10.1016/j.neuron.2009.08.025 · Cited by 352 · see citing works
An fMRI study of reward-related probability learning
Delgado, M. R., Miller, M. M., Inati, S., & Phelps, E. A. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.10.002 · Cited by 337 · see citing works
Linking nucleus accumbens dopamine and blood oxygenation
Knutson, B., & Gibbs, S. E. B. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0686-7 · Cited by 337 · see citing works
Frey, U., Schroeder, H., & Matthies, H. (1990) · Brain Research · doi:10.1016/0006-8993(90)91578-5 · Cited by 333 · see citing works
Gasbarri, A., Verney, C., Innocenzi, R., Campana, E., & Pacitti, C. (1994) · Brain Research · doi:10.1016/0006-8993(94)90512-6 · Cited by 302 · see citing works
Distinct Value Signals in Anterior and Posterior Ventromedial Prefrontal Cortex
Smith, D. V., Hayden, B. Y., Truong, T.-K., Song, A. W., Platt, M. L., & Huettel, S. A. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.3319-09.2010 · Cited by 300 · see citing works
Hippocampal contribution to the novel use of relational information in declarative memory
Preston, A. R., Shrager, Y., Dudukovic, N. M., & Gabrieli, J. D. E. (2004) · Hippocampus · doi:10.1002/hipo.20009 · Cited by 296 · see citing works
Connectivity-based segregation of the human striatum predicts personality characteristics
Cohen, M. X., Schoene‐Bake, J., Elger, C. E., & Weber, B. (2008) · Nature Neuroscience · doi:10.1038/nn.2228 · Cited by 281 · see citing works
Striatal activity underlies novelty-based choice in humans
Wittmann, B. C., Daw, N. D., Seymour, B., & Dolan, R. J. (2008) · Neuron · doi:10.1016/j.neuron.2008.04.027 · Cited by 277 · see citing works
Self-citationMind the Gap: Binding Experiences across Space and Time in the Human Hippocampus
Staresina, B. P., & Davachi, L. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.024 · Cited by 273 · see citing works
Neural signature of fictive learning signals in a sequential investment task
Lohrenz, T. M., McCabe, K., Camerer, C. F., & Montague, P. R. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0608842104 · Cited by 263 · see citing works
Tracking the emergence of conceptual knowledge during human decision making
Kumaran, D., Summerfield, J. J., Hassabis, D., & Maguire, E. A. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.030 · Cited by 260 · see citing works
Integrating hippocampus and striatum in decision-making
Johnson, A. C., van der Meer, M. A. A., & Redish, A. D. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.01.003 · Cited by 254 · see citing works
Dissociating Hippocampal versus Basal Ganglia Contributions to Learning and Transfer
Myers, C. E., Shohamy, D., Gluck, M. A., Grossman, S., Kluger, A., Ferris, S., Golomb, J., Schnirman, G., & Schwartz, R. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321208123 · Cited by 243 · see citing works
Zeithamová, D., & Preston, A. R. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.3250-10.2010 · Cited by 232 · see citing works
Resistance to forgetting associated with hippocampus-mediated reactivation during new learning
Kuhl, B. A., Shah, A. T., DuBrow, S., & Wagner, A. D. (2010) · Nature Neuroscience · doi:10.1038/nn.2498 · Cited by 225 · see citing works
Basal ganglia and dopamine contributions to probabilistic category learning
Daphna Shohamy, A., Myers, C. E., Kalanithi, J., & Gluck, M. A. (2007) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2007.07.008 · Cited by 216 · see citing works
van der Meer, M. A. A., Johnson, A. C., Schmitzer-Torbert, N. C., & Redish, A. D. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.023 · Cited by 210 · see citing works
LINEAR-NONLINEAR-POISSON MODELS OF PRIMATE CHOICE DYNAMICS
Corrado, G. S., Sugrue, L. P., Seung, H. S., & Newsome, W. T. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.23-05 · Cited by 206 · see citing works
A Pallidus-Habenula-Dopamine Pathway Signals Inferred Stimulus Values
Bromberg-Martin, E. S., Matsumoto, M., Hong, S., & Hikosaka, O. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00158.2010 · Cited by 182 · see citing works
Neural Correlates of Forward Planning in a Spatial Decision Task in Humans
Simon, D. A., & Daw, N. D. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.4647-10.2011 · Cited by 181 · see citing works
Self-citationl-dopa impairs learning, but spares generalization, in Parkinson's disease
Daphna Shohamy, A., Myers, C. E., Geghman, K. D., Sage, J., & Gluck, M. A. (2005) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2005.07.013 · Cited by 152 · see citing works
Acquired equivalence and distinctiveness of cues.
Honey, R. C., & Hall, G. (1989) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.15.4.338 · Cited by 151 · see citing works
Signals in Human Striatum Are Appropriate for Policy Update Rather than Value Prediction
Li, J., & Daw, N. D. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.6316-10.2011 · Cited by 149 · see citing works
Self-citationAn fMRI Analysis of the Human Hippocampus: Inference, Context, and Task Awareness
Greene, A. J., Gross, W. L., Elsinger, C. L., & Rao, S. M. (2006) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2006.18.7.1156 · Cited by 124 · see citing works
Dissociating hippocampal and striatal contributions to sequential prediction learning
Bornstein, A. M., & Daw, N. D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07920.x · Cited by 120 · see citing works
Self-citationGershman, S. J., Pesaran, B., & Daw, N. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2469-09.2009 · Cited by 118 · see citing works
Self-citationAcquired equivalence and distinctiveness of cues.
Honey, R. C., & Hall, G. (1989) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.15.4.338 · Cited by 111 · see citing works
The Cognitive Neuroscience of Motivation and Learning
Daw, N. D., & Daphna Shohamy, A. (2008) · Social Cognition · doi:10.1521/soco.2008.26.5.593 · Cited by 103 · see citing works
Self-citationParallel contributions of distinct human memory systems during probabilistic learning
Dickerson, K. C., Li, J., & Delgado, M. R. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.10.080 · Cited by 92 · see citing works
Brain Hemispheres Selectively Track the Expected Value of Contralateral Options
Palminteri, S., Boraud, T., Lafargue, G., Dubois, B., & Pessiglione, M. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.1500-09.2009 · Cited by 91 · see citing works
Acquired equivalence and distinctiveness of cues.
Honey, R. C., & Hall, G. (1989) · PubMed · Cited by 88 · see citing works
Complementary memory systems: competition, cooperation and compensation
Hartley, T., & Burgess, N. (2005) · Trends in Neurosciences · doi:10.1016/j.tins.2005.02.004 · Cited by 87 · see citing works
Schönberg, T., O’Doherty, J. P., Joel, D., Inzelberg, R., Segev, Y., & Daw, N. D. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.08.011 · Cited by 81 · see citing works
Self-citationAcquired equivalence and distinctiveness of cues: II. Neural manipulations and their implications.
Coutureau, É., Killcross, S., Good, M. A., Marshall, V. J., Ward-Robinson, J., & Honey, R. C. (2002) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.28.4.388 · Cited by 65 · see citing works
Moustafa, A. A., Myers, C. E., & Gluck, M. A. (2009) · Brain Research · doi:10.1016/j.brainres.2009.04.020 · Cited by 55 · see citing works
Striatal and Medial Temporal Lobe Functional Interactions during Visuomotor Associative Learning
Mattfeld, A. T., & Stark, C. E. L. (2010) · Cerebral Cortex · doi:10.1093/cercor/bhq144 · Cited by 49 · see citing works
ON THE CELLULAR LOCALIZATION OF CATECHOL AMINES IN THE BRAIN
FALCK, B., & Hillarp, N. (2008) · Acta Anatomica · doi:10.1159/000141530 · Cited by 40 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Balleine B Daw N O'Doherty JP Multiple forms of value learning and the function of dopamine 2008 Elsevier
- Eichenbaum H Cohen NJ From Conditioning to Conscious Recollection: Memory Systems of the Brain 2001 New York Oxford University Press
Cited by
Sorted by citations
Informatic parcellation of the network involved in the computation of subjective value
Clithero, J. A., & Rangel, A. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst106 · Cited by 706 · see citing works
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380 · see citing works
Self-citationProcess Account of Curiosity and Interest: A Reward-Learning Perspective
Murayama, K., FitzGibbon, L., & Sakaki, M. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09499-9 · Cited by 316 · see citing works
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280 · see citing works
Representation of aversive prediction errors in the human periaqueductal gray
Roy, M., Shohamy, D., Daw, N., Jepma, M., Wimmer, G. E., & Wager, T. D. (2014) · Nature Neuroscience · doi:10.1038/nn.3832 · Cited by 246 · see citing works
Self-citationTowards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224 · see citing works
Identity-specific coding of future rewards in the human orbitofrontal cortex
Howard, J. D., Gottfried, J. A., Tobler, P. N., & Kahnt, T. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1503550112 · Cited by 222 · see citing works
The algorithmic architecture of exploration in the human brain
Schulz, E., & Gershman, S. J. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2018.11.003 · Cited by 184 · see citing works
Influence of reward motivation on human declarative memory
Lombard, E., Bavelier, D., & Schwartz, S. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.11.015 · Cited by 174 · see citing works
Decision Making: From Neuroscience to Psychiatry
Lee, D. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.008 · Cited by 169 · see citing works
Kumaran, D., Banino, A., Blundell, C., Hassabis, D., & Dayan, P. (2016) · Neuron · doi:10.1016/j.neuron.2016.10.052 · Cited by 155 · see citing works
Instrumental learning of traits versus rewards: dissociable neural correlates and effects on choice
Hackel, L. M., Doll, B. B., & Amodio, D. M. (2015) · Nature Neuroscience · doi:10.1038/nn.4080 · Cited by 148 · see citing works
Baram, A. B., Müller, T., Nili, H., Garvert, M. M., & Behrens, T. E. J. (2020) · Neuron · doi:10.1016/j.neuron.2020.11.024 · Cited by 127 · see citing works
Alderson‐Day, B., Diederen, K. M. J., Fernyhough, C., Ford, J. M., Horga, G., Margulies, D. S., McCarthy‐Jones, S., Northoff, G., Shine, J. M., Turner, J. A., et al. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbw078 · Cited by 129 · see citing works
Alós‐Ferrer, C., Hügelschäfer, S., & Li, J. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00169 · Cited by 130 · see citing works
Beyond dichotomies in reinforcement learning
Collins, A. G. E., & Cockburn, J. (2020) · Nature reviews. Neuroscience · doi:10.1038/s41583-020-0355-6 · Cited by 128 · see citing works
Episodic Memory Encoding Interferes with Reward Learning and Decreases Striatal Prediction Errors
Wimmer, G. E., Braun, E. K., Daw, N. D., & Daphna Shohamy, A. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0204-14.2014 · Cited by 126 · see citing works
Self-citationOrbitofrontal neurons infer the value and identity of predicted outcomes
Stalnaker, T. A., Cooch, N. K., McDannald, M. A., Liu, T., Wied, H. M., & Schoenbaum, G. M. (2014) · Nature Communications · doi:10.1038/ncomms4926 · Cited by 123 · see citing works
Samborska, V., Butler, J. L., Walton, M. E., Behrens, T. E. J., & Akam, T. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01149-8 · Cited by 133 · see citing works
Structured, uncertainty-driven exploration in real-world consumer choice
Schulz, E., Bhui, R., Love, B. C., Brier, B., Todd, M. T., & Gershman, S. J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821028116 · Cited by 111 · see citing works
Using fMRI to study reward processing in humans: past, present, and future
Wang, K. S., Smith, D. V., & Delgado, M. R. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00333.2015 · Cited by 105 · see citing works
Dynamic Flexibility in Striatal-Cortical Circuits Supports Reinforcement Learning
Gerraty, R. T., Davidow, J. Y., Foerde, K., Galván, A., Bassett, D. S., & Daphna Shohamy, A. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.2084-17.2018 · Cited by 102 · see citing works
Reiter, A. M. F., Heinze, H., Schlagenhauf, F., & Deserno, L. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2016.95 · Cited by 103 · see citing works
Cognitive Neurostimulation: Learning to Volitionally Sustain Ventral Tegmental Area Activation
MacInnes, J., Dickerson, K. C., Chen, N., & Adcock, R. A. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.002 · Cited by 98 · see citing works
Multiple memory systems as substrates for multiple decision systems
Doll, B. B., Shohamy, D., & Daw, N. D. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.04.014 · Cited by 92 · see citing works
Self-citationBornstein, A. M., & Daw, N. D. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003387 · Cited by 89 · see citing works
Self-citationForming attitudes via neural activity supporting affective episodic simulations
Benoit, R. G., Paulus, P. C., & Schacter, D. L. (2019) · Nature Communications · doi:10.1038/s41467-019-09961-w · Cited by 90 · see citing works
Normative development of ventral striatal resting state connectivity in humans
Fareri, D. S., Gabard-Durnam, L. J., Goff, B., Flannery, J., Gee, D. G., Lumian, D. S., Caldera, C., & Tottenham, N. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.06.022 · Cited by 85 · see citing works
Konovalov, A., & Krajbich, I. M. (2016) · Nature Communications · doi:10.1038/ncomms12438 · Cited by 86 · see citing works
Gerraty, R. T., Davidow, J. Y., Wimmer, G. E., Kahn, I., & Daphna Shohamy, A. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0185-14.2014 · Cited by 78 · see citing works
Self-citationHippocampal spatio-predictive cognitive maps adaptively guide reward generalization
Garvert, M. M., Saanum, T., Schulz, E., Schuck, N. W., & Doeller, C. F. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01283-x · Cited by 82 · see citing works
Reiter, A. M. F., Deserno, L., Kallert, T. W., Heinze, H., Heinz, A., & Schlagenhauf, F. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.4322-15.2016 · Cited by 83 · see citing works
Goal-oriented and habitual decisions: Neural signatures of model-based and model-free learning
Huang, Y., Yaple, Z. A., & Yu, R. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116834 · Cited by 79 · see citing works
Hippocampal pattern separation supports reinforcement learning
Ballard, I. C., Wagner, A. D., & McClure, S. M. (2019) · Nature Communications · doi:10.1038/s41467-019-08998-1 · Cited by 65 · see citing works
Finding structure in multi-armed bandits
Schulz, E., Franklin, N. T., & Gershman, S. J. (2018) · Cognitive Psychology · doi:10.1016/j.cogpsych.2019.101261 · Cited by 59 · see citing works
Neural knowledge assembly in humans and neural networks
Nelli, S., Braun, L., Dumbalska, T., Saxe, A., & Summerfield, C. (2023) · Neuron · doi:10.1016/j.neuron.2023.02.014 · Cited by 57 · see citing works
Striatal dopamine, reward, and decision making in schizophrenia
Deserno, L., Schlagenhauf, F., & Heinz, A. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/ldeserno · Cited by 58 · see citing works
Fonzo, G. A. (2018) · Neurobiology of Stress · doi:10.1016/j.ynstr.2018.10.002 · Cited by 56 · see citing works
Wimmer, G. E., Li, J. K., Gorgolewski, K. J., & Poldrack, R. A. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.0075-18.2018 · Cited by 53 · see citing works
Self-citationDopamine-independent effect of rewards on choices through hidden-state inference
Blanco-Pozo, M., Akam, T. E., & Walton, M. E. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01542-x · Cited by 70 · see citing works
Quevedo, K., Ng, R., Scott, H., Kodavaganti, S., Smyda, G., Diwadkar, V., & Phillips, M. (2017) · Biological Psychology · doi:10.1016/j.biopsycho.2016.11.004 · Cited by 53 · see citing works
Social Cognition as Reinforcement Learning: Feedback Modulates Emotion Inference
Zaki, J., Kallman, S., Wimmer, G. E., Ochsner, K. N., & Daphna Shohamy, A. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00978 · Cited by 51 · see citing works
Self-citationReinforcement learning, efficient coding, and the statistics of natural tasks
Botvinick, M., Weinstein, A., Solway, A., & Barto, A. G. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.08.009 · Cited by 50 · see citing works
How Glitter Relates to Gold: Similarity-Dependent Reward Prediction Errors in the Human Striatum
Kahnt, T., Park, S. Q., Burke, C. J., & Tobler, P. N. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.2383-12.2012 · Cited by 51 · see citing works
Dopamine regulates stimulus generalization in the human hippocampus
Kahnt, T., & Tobler, P. N. (2016) · eLife · doi:10.7554/elife.12678 · Cited by 44 · see citing works
Baudonnat, M., Huber, A., David, V., & Walton, M. E. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00175 · Cited by 42 · see citing works
The feedback‐related negativity indexes prediction error in active but not observational learning
Burnside, R., Fischer, A. G., & Ullsperger, M. (2019) · Psychophysiology · doi:10.1111/psyp.13389 · Cited by 43 · see citing works
Changes in corticostriatal connectivity during reinforcement learning in humans
Horga, G., Maia, T. V., Marsh, R., Hao, X., Xu, D., Duan, Y., Tau, G. Z., Graniello, B., Wang, Z., Kangarlu, A., et al. (2014) · Human Brain Mapping · doi:10.1002/hbm.22665 · Cited by 42 · see citing works
Humans can navigate complex graph structures acquired during latent learning
Rmus, M., Ritz, H., Hunter, L. E., Bornstein, A. M., & Shenhav, A. (2019) · Cognition · doi:10.1016/j.cognition.2022.105103 · Cited by 40 · see citing works
Distinct replay signatures for prospective decision-making and memory preservation
Wimmer, G. E., Liu, Y., McNamee, D., & Dolan, R. J. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2205211120 · Cited by 46 · see citing works
Self-citationO’Doherty, J. P. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08074.x · Cited by 36 · see citing works
Zhang, S., Hu, S., Chao, H. H., & Li, C.-S. R. (2017) · Brain Structure and Function · doi:10.1007/s00429-016-1358-y · Cited by 38 · see citing works
Realigning Models of Habitual and Goal-Directed Decision-Making
Miller, K. J., Ludvig, E. A., Pezzulo, G., & Shenhav, A. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812098-9.00018-8 · Cited by 41 · see citing works
Attentional modulation of reward processing in the human brain
Rothkirch, M., Schmack, K., Deserno, L., Darmohray, D. M., & Sterzer, P. (2013) · Human Brain Mapping · doi:10.1002/hbm.22383 · Cited by 38 · see citing works
Computational approaches to schizophrenia: A perspective on negative symptoms
Deserno, L., Heinz, A., & Schlagenhauf, F. (2017) · Schizophrenia Research · doi:10.1016/j.schres.2016.10.004 · Cited by 31 · see citing works
Modulating the Use of Multiple Memory Systems in Value-based Decisions with Contextual Novelty
Duncan, K. D., Semmler, A., & Daphna Shohamy, A. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01447 · Cited by 30 · see citing works
Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia
Palombo, D. J., Hayes, S. M., Reid, A. G., & Verfaellie, M. H. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-018-00687-8 · Cited by 30 · see citing works
Frank, L. E., Preston, A. R., & Zeithamová, D. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-019-00700-8 · Cited by 27 · see citing works
Unifying Principles of Generalization: Past, Present, and Future
Wu, C. M., Meder, B., & Schulz, E. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-021524-110810 · Cited by 33 · see citing works
Inference and Search on Graph-Structured Spaces
Wu, C. M., Schulz, E., & Gershman, S. J. (2020) · Computational Brain & Behavior · doi:10.1007/s42113-020-00091-x · Cited by 26 · see citing works
Moustafa, A. A., Hewedi, D. H., Eissa, A. M., Myers, C. E., & Sadek, H. A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0046496 · Cited by 26 · see citing works
Samborska, V., Butler, J. L., Walton, M. E., Behrens, T. E. J., & Akam, T. E. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.03.05.433967 · Cited by 25 · see citing works
Rapid encoding of task regularities in the human hippocampus guides sensorimotor timing
Polti, I., Nau, M., Kaplan, R., van Wassenhove, V., & Doeller, C. F. (2022) · eLife · doi:10.7554/elife.79027 · Cited by 23 · see citing works
Abir, Y., Marvin, C. B., van Geen, C., Leshkowitz, M., Hassin, R. R., & Shohamy, D. (2022) · Nature Communications · doi:10.1038/s41467-022-30011-5 · Cited by 22 · see citing works
Wimmer, G. E., & Poldrack, R. A. (2021) · Memory & Cognition · doi:10.3758/s13421-021-01233-7 · Cited by 24 · see citing works
Self-citationPain and the PAG: learning from painful mistakes
Eippert, F., & Tracey, I. (2014) · Nature Neuroscience · doi:10.1038/nn.3844 · Cited by 21 · see citing works
Value restructures the organization of free recall
Horwath, E. A., Rouhani, N., DuBrow, S., & Murty, V. P. (2022) · Cognition · doi:10.1016/j.cognition.2022.105315 · Cited by 21 · see citing works
Schlegelmilch, R., & von Helversen, B. (2020) · Journal of Experimental Psychology General · doi:10.1037/xge0000747 · Cited by 19 · see citing works
Action selection in multi-effector decision making
Madlon‐Kay, S., Pesaran, B., & Daw, N. D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.12.001 · Cited by 16 · see citing works
Self-citationBursts and Heavy Tails in Temporal and Sequential Dynamics of Foraging Decisions
Jung, K., Jang, H., Kralik, J. D., & Jeong, J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003759 · Cited by 14 · see citing works
The reinforcement heuristic in normal form games
Alós‐Ferrer, C., & Ritschel, A. (2018) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2018.06.014 · Cited by 15 · see citing works
Ma, S., Li, C. R., Zhang, S., Worhunsky, P. D., Zhou, N., Zhang, J., Liu, L., Yao, Y., & Fang, X. (2021) · Journal of Behavioral Addictions · doi:10.1556/2006.2021.00010 · Cited by 15 · see citing works
van de Vijver, I., & Ligneul, R. (2019) · Aging Neuropsychology and Cognition · doi:10.1080/13825585.2019.1664389 · Cited by 14 · see citing works
Generalization of placebo pain relief
Kampermann, L., Tinnermann, A., & Büchel, C. (2020) · Pain · doi:10.1097/j.pain.0000000000002166 · Cited by 13 · see citing works
Brändle, F., Binz, M., & Schulz, E. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009026949.008 · Cited by 12 · see citing works
Reiter, A. M. F., Koch, S., Schröger, E., Hinrichs, H., Heinze, H., Deserno, L., & Schlagenhauf, F. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00957 · Cited by 11 · see citing works
A one-bit, comparison-based gradient estimator
Cai, H., McKenzie, D. H., Yin, W., & Zhang, Z. (2022) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2022.03.003 · Cited by 10 · see citing works
Dopamine-independent state inference mediates expert reward guided decision makingPreprint
Blanco-Pozo, M., Akam, T. E., & Walton, M. E. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.06.25.449995 · Cited by 10 · see citing works
Case-Based Decision Neuroscience
Bhui, R. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812098-9.00004-8 · Cited by 10 · see citing works
Efficient coding of cognitive variables underlies dopamine response and choice behavior
Motiwala, A., Soares, S., Atallah, B. V., Paton, J. J., & Machens, C. K. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01085-7 · Cited by 9 · see citing works
Comparing experience- and description-based economic preferences across 11 countries
Anlló, H., Bavard, S., Benmarrakchi, F., Bonagura, D., Cerrotti, F., Cicue, M., Gueguen, M. C. M., Guzmán, E., Kadieva, D., Kobayashi, M., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01894-9 · Cited by 9 · see citing works
Colas, J. T., Dundon, N. M., Gerraty, R. T., Saragosa-Harris, N. M., Szymula, K. P., Tanwisuth, K., Tyszka, J. M., van Geen, C., Ju, H., Toga, A. W., et al. (2022) · Human Brain Mapping · doi:10.1002/hbm.25988 · Cited by 8 · see citing works
Kéri, S., Nagy, H., Levy‐Gigi, E., & Kelemen, O. (2013) · European Journal of Neuroscience · doi:10.1111/ejn.12350 · Cited by 8 · see citing works
An enhanced PI controller based on adaptive iterative learning control
Liang, H., Yang, C., Lv, M., Sun, B., & Li, Y. (2023) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.6940 · Cited by 7 · see citing works
Neural knowledge assembly in humans and deep networksPreprint
Nelli, S., Braun, L., Dumbalska, T., Saxe, A., & Summerfield, C. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.10.21.465374 · Cited by 7 · see citing works
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6 · see citing works
Pereira, A. R., Alemi, M., Cerqueira-Nunes, M., Monteiro, C., Galhardo, V., & Cardoso-Cruz, H. (2023) · Neurology International · doi:10.3390/neurolint15040082 · Cited by 5 · see citing works
May, B., Palace, M., Milne, R. J., Dalton, G., Meenaghan, A., & Terbeck, S. (2025) · Cognition Technology & Work · doi:10.1007/s10111-025-00815-8 · Cited by 4 · see citing works
Lombard, E., Aberg, K. C., Bavelier, D., & Schwartz, S. (2020) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01660 · Cited by 4 · see citing works
Unifying principles of generalization: past, present, and futurePreprint
Wu, C. M., Meder, B., & Schulz, E. (2024) · doi:10.31234/osf.io/6uz9q · Cited by 4 · see citing works
Rigoli, F. (2025) · Psychonomic Bulletin & Review · doi:10.3758/s13423-025-02813-3 · Cited by 3 · see citing works
Distinct replay signatures for prospective decision-making and memory preservationPreprint
Wimmer, G. E., Liu, Y., McNamee, D., & Dolan, R. J. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.08.467745 · Cited by 3 · see citing works
Self-citationAdapting Zeroth Order Algorithms for Comparison-Based Optimization
Slavin, I. (2023) · SIAM Undergraduate Research Online · doi:10.1137/22s1530951 · Cited by 2 · see citing works
Hippocampal spatio-temporal cognitive maps adaptively guide reward generalizationPreprint
Garvert, M. M., Saanum, T., Schulz, E., Schuck, N. W., & Doeller, C. F. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.10.22.465012 · Cited by 2 · see citing works
Individual curiosity modulates exploration in sequential book selection
Gong, X., Boorman, E. D., Shen, C., & Huskey, R. (2026) · PNAS Nexus · doi:10.1093/pnasnexus/pgag226 · Cited by 1 · see citing works
Maor, I., Atwell, J. E., Ascher, I., Zhao, Y., Takahashi, Y. K., Hart, E. E., Pereira, F., & Schoenbaum, G. M. (2026) · Nature Communications · doi:10.1038/s41467-026-69330-2 · Cited by 1 · see citing works
Exploration in the wildPreprint
Schulz, E., Bhui, R., Love, B. C., Brier, B., Todd, M. T., & Gershman, S. J. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/492058 · Cited by 1 · see citing works
Anlló, H., & Palminteri, S. (2023) · doi:10.31234/osf.io/pvycs · Cited by 1 · see citing works
Reactivation of reward representations associated with the reinforcement of behaviorPreprint
Tong, A. P., Sreekumar, V., Inati, S. K., Zaghloul, K. A., & Woolrich, M. W. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.23.628672 · Cited by 1 · see citing works
Joint models reveal human subcortical underpinnings of choice and learning behaviourPreprint
Miletić, S., Stevenson, N., Bazin, P., Alkemade, A., Isherwood, S. J., Trutti, A. C., Tse, D. H. Y., Haberg, A. K., & Forstmann, B. U. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.20.629647 · Cited by 1 · see citing works
- Authors
- Wimmer, G. E., Daw, N. D., & Shohamy, D.
- Affiliations
- Columbia University (US)
- New York University (US)
- Venue
- European Journal of Neuroscience
- Publisher
- Wiley
- Published
- 2012-04
- Volume
- 35
- Issue
- 7
- Pages
- 1092-1104
- ISSN
- 0953-816X (ISSN-L)
- 1460-9568
- PMC
- PMC3404618
- OpenAlex
- W1535087671
- Cited by
- 111
- References
- 107
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 7 · 2023: 8 · 2022: 9 · 2021: 9 · 2020: 8 · 2019: 11 · 2018: 9 · 2017: 1 · 2016: 14 · 2015: 5 · 2014: 9 · 2013: 6 · 2012: 5
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- European Journal of Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health (1r03da026957-01)
- National Institute on Drug Abuse (R03-DA026957)
- National Institute of Neurological Disorders and Stroke (R01NS078784)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- reinforcement learning, hippocampus, ventral striatum, relational learning, functional connectivity, computational modeling, choice behavior, reward probability, dopamine
- MeSH terms
- Adolescent, Adult, Basal Ganglia, Female, Generalization, Psychological, Humans, Learning, Male, Photic Stimulation, Psychomotor Performance, Reinforcement, Psychology, Reward, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
