The DOI matches its Crossref record; 107 of the 107 references it cites were found in a registry or an index.
Generalization of value in reinforcement learning by humans
Open accessAbstract
Research in decision‐making has focused on the role of dopamine and its striatal targets in guiding choices via learned stimulus–reward or stimulus–response associations, behavior that is well described by reinforcement learning theories. However, basic reinforcement learning is relatively limited in scope and does not explain how learning about stimulus regularities or relations may guide decision‐making. A candidate mechanism for this type of learning comes from the domain of memory, which has highlighted a role for the hippocampus in learning of stimulus–stimulus relations, typically dissociated from the role of the striatum in stimulus–response learning. Here, we used functional magnetic resonance imaging and computational model‐based analyses to examine the joint contributions of these mechanisms to reinforcement learning. Humans performed a reinforcement learning task with added relational structure, modeled after tasks used to isolate hippocampal contributions to memory. On each trial participants chose one of four options, but the reward probabilities for pairs of options were correlated across trials. This (uninstructed) relationship between pairs of options potentially enabled an observer to learn about option values based on experience with the other options and to generalize across them. We observed blood oxygen level‐dependent (BOLD) activity related to learning in the striatum and also in the hippocampus. By comparing a basic reinforcement learning model to one augmented to allow feedback to generalize between correlated options, we tested whether choice behavior and BOLD activity were influenced by the opportunity to generalize across correlated options. Although such generalization goes beyond standard computational accounts of reinforcement learning and striatal BOLD, both choices and striatal BOLD activity were better explained by the augmented model. Consistent with the hypothesized role for the hippocampus in this generalization, functional connectivity between the ventral striatum and hippocampus was modulated, across participants, by the ability of the augmented model to capture participants’ choice. Our results thus point toward an interactive model in which striatal reinforcement learning systems may employ relational representations typically associated with the hippocampus.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As expected, in contradistinction to the main study, appetitive PEs to monetary rewards were tracked by activity in the VS. (t(20) = 5.77, p < 0.001), but not the PAG (PAG–appetitive: t(20) = 1.54, p = 0.14; see Supplementary Fig. 2).”
From Representation of aversive prediction errors in the human periaqueductal gray (2014) · cites this paper as [23]
In the citing paper
“As expected, in contradistinction to the main study, appetitive PEs to monetary rewards were tracked by activity in the VS. (t(20) = 5.77, p < 0.001), but not the PAG (PAG–appetitive: t(20) = 1.54, p = 0.14; see Supplementary Fig. 2).”
From Representation of aversive prediction errors in the human periaqueductal gray (2014) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 20,012
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., & Joliot, M. (2002) · NeuroImage · doi:10.1006/nimg.2001.0978 · Cited by 16,319
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Cragg, J. G., & Zarembka, P. (1975) · Canadian Journal of Economics/Revue canadienne d économique · doi:10.2307/134075 · Cited by 8,477
Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans.
Squire, L. R. (1992) · Psychological Review · doi:10.1037/0033-295x.99.2.195 · Cited by 5,376
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785
Psychophysiological and Modulatory Interactions in Neuroimaging
Friston, K. J., Buechel, C., Fink, G. R., Morris, J., Rolls, E., & Dolan, R. J. (1997) · NeuroImage · doi:10.1006/nimg.1997.0291 · Cited by 3,097
Research works printed but not linked: 2
- Eichenbaum H., 2001, From Conditioning to Conscious Recollection: Memory Systems of the Brain
- Thorndike E.L., 1911, Animal Intelligence
Cited by
Sorted by citations
Informatic parcellation of the network involved in the computation of subjective value
Clithero, J. A., & Rangel, A. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst106 · Cited by 706
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380
Self-citationProcess Account of Curiosity and Interest: A Reward-Learning Perspective
Murayama, K., FitzGibbon, L., & Sakaki, M. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09499-9 · Cited by 334
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280
Representation of aversive prediction errors in the human periaqueductal gray
Roy, M., Shohamy, D., Daw, N., Jepma, M., Wimmer, G. E., & Wager, T. D. (2014) · Nature Neuroscience · doi:10.1038/nn.3832 · Cited by 246
Self-citationTowards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224
Identity-specific coding of future rewards in the human orbitofrontal cortex
Howard, J. D., Gottfried, J. A., Tobler, P. N., & Kahnt, T. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1503550112 · Cited by 231
The algorithmic architecture of exploration in the human brain
Schulz, E., & Gershman, S. J. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2018.11.003 · Cited by 184
Influence of reward motivation on human declarative memory
Lombard, E., Bavelier, D., & Schwartz, S. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.11.015 · Cited by 191
Decision Making: From Neuroscience to Psychiatry
Lee, D. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.008 · Cited by 178
Kumaran, D., Banino, A., Blundell, C., Hassabis, D., & Dayan, P. (2016) · Neuron · doi:10.1016/j.neuron.2016.10.052 · Cited by 163
Instrumental learning of traits versus rewards: dissociable neural correlates and effects on choice
Hackel, L. M., Doll, B. B., & Amodio, D. M. (2015) · Nature Neuroscience · doi:10.1038/nn.4080 · Cited by 148
Baram, A. B., Müller, T., Nili, H., Garvert, M. M., & Behrens, T. E. J. (2020) · Neuron · doi:10.1016/j.neuron.2020.11.024 · Cited by 134
Alderson‐Day, B., Diederen, K. M. J., Fernyhough, C., Ford, J. M., Horga, G., Margulies, D. S., McCarthy‐Jones, S., Northoff, G., Shine, J. M., Turner, J. A., et al. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbw078 · Cited by 132
Alós‐Ferrer, C., Hügelschäfer, S., & Li, J. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00169 · Cited by 142
Beyond dichotomies in reinforcement learning
Collins, A. G. E., & Cockburn, J. (2020) · Nature reviews. Neuroscience · doi:10.1038/s41583-020-0355-6 · Cited by 147
Episodic Memory Encoding Interferes with Reward Learning and Decreases Striatal Prediction Errors
Wimmer, G. E., Braun, E. K., Daw, N. D., & Shohamy, D. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0204-14.2014 · Cited by 134
Self-citationOrbitofrontal neurons infer the value and identity of predicted outcomes
Stalnaker, T. A., Cooch, N. K., McDannald, M. A., Liu, T., Wied, H. M., & Schoenbaum, G. M. (2014) · Nature Communications · doi:10.1038/ncomms4926 · Cited by 124
Samborska, V., Butler, J. L., Walton, M. E., Behrens, T. E. J., & Akam, T. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01149-8 · Cited by 141
Structured, uncertainty-driven exploration in real-world consumer choice
Schulz, E., Bhui, R., Love, B. C., Brier, B., Todd, M. T., & Gershman, S. J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821028116 · Cited by 111
Using fMRI to study reward processing in humans: past, present, and future
Wang, K. S., Smith, D. V., & Delgado, M. R. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00333.2015 · Cited by 105
Dynamic Flexibility in Striatal-Cortical Circuits Supports Reinforcement Learning
Gerraty, R. T., Davidow, J. Y., Foerde, K., Galván, A., Bassett, D. S., & Shohamy, D. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.2084-17.2018 · Cited by 102
Reiter, A. M. F., Heinze, H., Schlagenhauf, F., & Deserno, L. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2016.95 · Cited by 104
Cognitive Neurostimulation: Learning to Volitionally Sustain Ventral Tegmental Area Activation
MacInnes, J., Dickerson, K. C., Chen, N., & Adcock, R. A. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.002 · Cited by 100
Multiple memory systems as substrates for multiple decision systems
Doll, B. B., Shohamy, D., & Daw, N. D. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.04.014 · Cited by 101
Self-citationBornstein, A. M., & Daw, N. D. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003387 · Cited by 96
Self-citationForming attitudes via neural activity supporting affective episodic simulations
Benoit, R. G., Paulus, P. C., & Schacter, D. L. (2019) · Nature Communications · doi:10.1038/s41467-019-09961-w · Cited by 95
Normative development of ventral striatal resting state connectivity in humans
Fareri, D. S., Gabard-Durnam, L. J., Goff, B., Flannery, J., Gee, D. G., Lumian, D. S., Caldera, C., & Tottenham, N. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.06.022 · Cited by 85
Konovalov, A., & Krajbich, I. M. (2016) · Nature Communications · doi:10.1038/ncomms12438 · Cited by 98
Gerraty, R. T., Davidow, J. Y., Wimmer, G. E., Kahn, I., & Shohamy, D. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0185-14.2014 · Cited by 80
Self-citationHippocampal spatio-predictive cognitive maps adaptively guide reward generalization
Garvert, M. M., Saanum, T., Schulz, E., Schuck, N. W., & Doeller, C. F. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01283-x · Cited by 90
Reiter, A. M. F., Deserno, L., Kallert, T. W., Heinze, H., Heinz, A., & Schlagenhauf, F. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.4322-15.2016 · Cited by 85
Goal-oriented and habitual decisions: Neural signatures of model-based and model-free learning
Huang, Y., Yaple, Z. A., & Yu, R. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116834 · Cited by 80
Hippocampal pattern separation supports reinforcement learning
Ballard, I. C., Wagner, A. D., & McClure, S. M. (2019) · Nature Communications · doi:10.1038/s41467-019-08998-1 · Cited by 67
Finding structure in multi-armed bandits
Schulz, E., Franklin, N. T., & Gershman, S. J. (2018) · Cognitive Psychology · doi:10.1016/j.cogpsych.2019.101261 · Cited by 57
Neural knowledge assembly in humans and neural networks
Nelli, S., Braun, L., Dumbalska, T., Saxe, A., & Summerfield, C. (2023) · Neuron · doi:10.1016/j.neuron.2023.02.014 · Cited by 63
Striatal dopamine, reward, and decision making in schizophrenia
Deserno, L., Schlagenhauf, F., & Heinz, A. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/ldeserno · Cited by 58
Fonzo, G. A. (2018) · Neurobiology of Stress · doi:10.1016/j.ynstr.2018.10.002 · Cited by 58
Wimmer, G. E., Li, J. K., Gorgolewski, K. J., & Poldrack, R. A. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.0075-18.2018 · Cited by 57
Self-citationDopamine-independent effect of rewards on choices through hidden-state inference
Blanco-Pozo, M., Akam, T., & Walton, M. E. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01542-x · Cited by 69
Quevedo, K., Ng, R., Scott, H., Kodavaganti, S., Smyda, G., Diwadkar, V., & Phillips, M. (2017) · Biological Psychology · doi:10.1016/j.biopsycho.2016.11.004 · Cited by 53
Social Cognition as Reinforcement Learning: Feedback Modulates Emotion Inference
Zaki, J., Kallman, S., Wimmer, G. E., Ochsner, K. N., & Shohamy, D. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00978 · Cited by 54
Self-citationReinforcement learning, efficient coding, and the statistics of natural tasks
Botvinick, M., Weinstein, A., Solway, A., & Barto, A. G. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.08.009 · Cited by 53
How Glitter Relates to Gold: Similarity-Dependent Reward Prediction Errors in the Human Striatum
Kahnt, T., Park, S. Q., Burke, C. J., & Tobler, P. N. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.2383-12.2012 · Cited by 51
Dopamine regulates stimulus generalization in the human hippocampus
Kahnt, T., & Tobler, P. N. (2016) · eLife · doi:10.7554/elife.12678 · Cited by 44
Baudonnat, M., Huber, A., David, V., & Walton, M. E. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00175 · Cited by 43
The feedback‐related negativity indexes prediction error in active but not observational learning
Burnside, R., Fischer, A. G., & Ullsperger, M. (2019) · Psychophysiology · doi:10.1111/psyp.13389 · Cited by 42
Changes in corticostriatal connectivity during reinforcement learning in humans
Horga, G., Maia, T. V., Marsh, R., Hao, X., Xu, D., Duan, Y., Tau, G. Z., Graniello, B., Wang, Z., Kangarlu, A., et al. (2014) · Human Brain Mapping · doi:10.1002/hbm.22665 · Cited by 40
Humans can navigate complex graph structures acquired during latent learning
Rmus, M., Ritz, H., Hunter, L. E., Bornstein, A. M., & Shenhav, A. (2019) · Cognition · doi:10.1016/j.cognition.2022.105103 · Cited by 42
Distinct replay signatures for prospective decision-making and memory preservation
Wimmer, G. E., Liu, Y., McNamee, D., & Dolan, R. J. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2205211120 · Cited by 47
Self-citationO’Doherty, J. P. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08074.x · Cited by 37
Zhang, S., Hu, S., Chao, H. H., & Li, C.-S. R. (2017) · Brain Structure and Function · doi:10.1007/s00429-016-1358-y · Cited by 38
Realigning Models of Habitual and Goal-Directed Decision-Making
Miller, K. J., Ludvig, E. A., Pezzulo, G., & Shenhav, A. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812098-9.00018-8 · Cited by 46
Attentional modulation of reward processing in the human brain
Rothkirch, M., Schmack, K., Deserno, L., Darmohray, D. M., & Sterzer, P. (2013) · Human Brain Mapping · doi:10.1002/hbm.22383 · Cited by 43
Computational approaches to schizophrenia: A perspective on negative symptoms
Deserno, L., Heinz, A., & Schlagenhauf, F. (2017) · Schizophrenia Research · doi:10.1016/j.schres.2016.10.004 · Cited by 31
Modulating the Use of Multiple Memory Systems in Value-based Decisions with Contextual Novelty
Duncan, K. D., Semmler, A., & Daphna Shohamy, A. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01447 · Cited by 30
Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia
Palombo, D. J., Hayes, S. M., Reid, A. G., & Verfaellie, M. H. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-018-00687-8 · Cited by 30
Frank, L. E., Preston, A. R., & Zeithamová, D. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-019-00700-8 · Cited by 27
Unifying Principles of Generalization: Past, Present, and Future
Wu, C. M., Meder, B., & Schulz, E. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-021524-110810 · Cited by 39
Inference and Search on Graph-Structured Spaces
Wu, C. M., Schulz, E., & Gershman, S. J. (2020) · Computational Brain & Behavior · doi:10.1007/s42113-020-00091-x · Cited by 26
Moustafa, A. A., Hewedi, D. H., Eissa, A. M., Myers, C. E., & Sadek, H. A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0046496 · Cited by 26
Samborska, V., Butler, J. L., Walton, M. E., Behrens, T. E. J., & Akam, T. E. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.03.05.433967 · Cited by 25
Rapid encoding of task regularities in the human hippocampus guides sensorimotor timing
Polti, I., Nau, M., Kaplan, R., van Wassenhove, V., & Doeller, C. F. (2022) · eLife · doi:10.7554/elife.79027 · Cited by 25
Abir, Y., Marvin, C. B., van Geen, C., Leshkowitz, M., Hassin, R. R., & Shohamy, D. (2022) · Nature Communications · doi:10.1038/s41467-022-30011-5 · Cited by 22
Wimmer, G. E., & Poldrack, R. A. (2021) · Memory & Cognition · doi:10.3758/s13421-021-01233-7 · Cited by 26
Self-citationPain and the PAG: learning from painful mistakes
Eippert, F., & Tracey, I. (2014) · Nature Neuroscience · doi:10.1038/nn.3844 · Cited by 21
Value restructures the organization of free recall
Horwath, E. A., Rouhani, N., DuBrow, S., & Murty, V. P. (2022) · Cognition · doi:10.1016/j.cognition.2022.105315 · Cited by 22
Schlegelmilch, R., & von Helversen, B. (2020) · Journal of Experimental Psychology General · doi:10.1037/xge0000747 · Cited by 21
Action selection in multi-effector decision making
Madlon‐Kay, S., Pesaran, B., & Daw, N. D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.12.001 · Cited by 17
Self-citationBursts and Heavy Tails in Temporal and Sequential Dynamics of Foraging Decisions
Jung, K., Jang, H., Kralik, J. D., & Jeong, J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003759 · Cited by 14
The reinforcement heuristic in normal form games
Alós‐Ferrer, C., & Ritschel, A. (2018) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2018.06.014 · Cited by 15
Ma, S., Li, C. R., Zhang, S., Worhunsky, P. D., Zhou, N., Zhang, J., Liu, L., Yao, Y., & Fang, X. (2021) · Journal of Behavioral Addictions · doi:10.1556/2006.2021.00010 · Cited by 15
van de Vijver, I., & Ligneul, R. (2019) · Aging Neuropsychology and Cognition · doi:10.1080/13825585.2019.1664389 · Cited by 14
Generalization of placebo pain relief
Kampermann, L., Tinnermann, A., & Büchel, C. (2020) · Pain · doi:10.1097/j.pain.0000000000002166 · Cited by 16
Brändle, F., Binz, M., & Schulz, E. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009026949.008 · Cited by 12
Reiter, A. M. F., Koch, S., Schröger, E., Hinrichs, H., Heinze, H., Deserno, L., & Schlagenhauf, F. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00957 · Cited by 11
A one-bit, comparison-based gradient estimator
Cai, H., McKenzie, D. H., Yin, W., & Zhang, Z. (2022) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2022.03.003 · Cited by 12
Dopamine-independent state inference mediates expert reward guided decision makingPreprint
Blanco-Pozo, M., Akam, T. E., & Walton, M. E. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.06.25.449995 · Cited by 10
Case-Based Decision Neuroscience
Bhui, R. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812098-9.00004-8 · Cited by 10
Efficient coding of cognitive variables underlies dopamine response and choice behavior
Motiwala, A., Soares, S., Atallah, B. V., Paton, J. J., & Machens, C. K. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01085-7 · Cited by 10
Comparing experience- and description-based economic preferences across 11 countries
Anlló, H., Bavard, S., Benmarrakchi, F., Bonagura, D., Cerrotti, F., Cicue, M., Gueguen, M. C. M., Guzmán, E., Kadieva, D., Kobayashi, M., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01894-9 · Cited by 9
Colas, J. T., Dundon, N. M., Gerraty, R. T., Saragosa‐Harris, N. M., Szymula, K. P., Tanwisuth, K., Tyszka, J. M., van Geen, C., Ju, H., Toga, A. W., et al. (2022) · Human Brain Mapping · doi:10.1002/hbm.25988 · Cited by 10
Kéri, S., Nagy, H., Levy‐Gigi, E., & Kelemen, O. (2013) · European Journal of Neuroscience · doi:10.1111/ejn.12350 · Cited by 8
An enhanced PI controller based on adaptive iterative learning control
Liang, H., Yang, C., Lv, M., Sun, B., & Li, Y. (2023) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.6940 · Cited by 7
Neural knowledge assembly in humans and deep networksPreprint
Nelli, S., Braun, L., Dumbalska, T., Saxe, A., & Summerfield, C. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.10.21.465374 · Cited by 7
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
Pereira, A. R., Alemi, M., Cerqueira-Nunes, M., Monteiro, C., Galhardo, V., & Cardoso-Cruz, H. (2023) · Neurology International · doi:10.3390/neurolint15040082 · Cited by 7
May, B., Palace, M., Milne, R. J., Dalton, G., Meenaghan, A., & Terbeck, S. (2025) · Cognition Technology & Work · doi:10.1007/s10111-025-00815-8 · Cited by 5
Lombard, E., Aberg, K. C., Bavelier, D., & Schwartz, S. (2020) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01660 · Cited by 4
Unifying principles of generalization: past, present, and futurePreprint
Wu, C. M., Meder, B., & Schulz, E. (2024) · doi:10.31234/osf.io/6uz9q · Cited by 4
Rigoli, F. (2025) · Psychonomic Bulletin & Review · doi:10.3758/s13423-025-02813-3 · Cited by 3
Distinct replay signatures for prospective decision-making and memory preservationPreprint
Wimmer, G. E., Liu, Y., McNamee, D., & Dolan, R. J. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.08.467745 · Cited by 3
Self-citationAdapting Zeroth Order Algorithms for Comparison-Based Optimization
Slavin, I. (2023) · SIAM Undergraduate Research Online · doi:10.1137/22s1530951 · Cited by 3
Hippocampal spatio-temporal cognitive maps adaptively guide reward generalizationPreprint
Garvert, M. M., Saanum, T., Schulz, E., Schuck, N. W., & Doeller, C. F. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.10.22.465012 · Cited by 2
Individual curiosity modulates exploration in sequential book selection
Gong, X., Boorman, E. D., Shen, C., & Huskey, R. (2026) · PNAS Nexus · doi:10.1093/pnasnexus/pgag226 · Cited by 1
Maor, I., Atwell, J. E., Ascher, I., Zhao, Y., Takahashi, Y. K., Hart, E. E., Pereira, F., & Schoenbaum, G. M. (2026) · Nature Communications · doi:10.1038/s41467-026-69330-2 · Cited by 1
Exploration in the wildPreprint
Schulz, E., Bhui, R., Love, B. C., Brier, B., Todd, M. T., & Gershman, S. J. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/492058 · Cited by 1
Anlló, H., & Palminteri, S. (2023) · doi:10.31234/osf.io/pvycs · Cited by 1
Reactivation of reward representations associated with the reinforcement of behaviorPreprint
Tong, A. P., Sreekumar, V., Inati, S. K., Zaghloul, K. A., & Woolrich, M. W. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.23.628672 · Cited by 1
Joint models reveal human subcortical underpinnings of choice and learning behaviourPreprint
Miletić, S., Stevenson, N., Bazin, P., Alkemade, A., Isherwood, S. J., Trutti, A. C., Tse, D. H. Y., Haberg, A. K., & Forstmann, B. U. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.20.629647 · Cited by 1
Metadata
- Authors
- Wimmer, G. E., Daw, N. D., & Shohamy, D.
- Affiliations
- Columbia University (US)
- New York University (US)
- Venue
- European Journal of Neuroscience
- Publisher
- Wiley
- Published
- 2012-04
- Volume
- 35
- Issue
- 7
- Pages
- 1092-1104
- ISSN
- 0953-816X (ISSN-L)
- 1460-9568
- PMC
- PMC3404618
- OpenAlex
- W1535087671
- Cited by
- 111
- References
- 107
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 7 · 2023: 8 · 2022: 9 · 2021: 9 · 2020: 8 · 2019: 11 · 2018: 9 · 2017: 1 · 2016: 14 · 2015: 5 · 2014: 9 · 2013: 6 · 2012: 5
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- European Journal of Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health (1r03da026957-01)
- National Institute on Drug Abuse (R03-DA026957)
- National Institute of Neurological Disorders and Stroke (R01NS078784)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- reinforcement learning, hippocampus, ventral striatum, relational learning, functional connectivity, computational modeling, choice behavior, reward probability, dopamine
- MeSH terms
- Adolescent, Adult, Basal Ganglia, Female, Generalization, Psychological, Humans, Learning, Male, Photic Stimulation, Psychomotor Performance, Reinforcement, Psychology, Reward, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
