The Tortoise and the Hare: Interactions between Reinforcement Learning and Working Memory
Open accessPublished version of a preprint
Abstract
Learning to make rewarding choices in response to stimuli depends on a slow but steady process, reinforcement learning, and a fast and flexible, but capacity-limited process, working memory. Using both systems in parallel, with their contributions weighted based on performance, should allow us to leverage the best of each system: rapid early learning, supplemented by long-term robust acquisition. However, this assumes that using one process does not interfere with the other. We use computational modeling to investigate the interactions between the two processes in a behavioral experiment and show that working memory interferes with reinforcement learning. Previous research showed that neural representations of reward prediction errors, a key marker of reinforcement learning, were blunted when working memory was used for learning. We thus predicted that arbitrating in favor of working memory to learn faster in simple problems would weaken the reinforcement learning process. We tested this by measuring performance in a delayed testing phase where the use of working memory was impossible, and thus participant choices depended on reinforcement learning. Counterintuitively, but confirming our predictions, we observed that associations learned most easily were retained worse than associations learned slower: Using working memory to learn quickly came at the cost of long-term retention. Computational modeling confirmed that this could only be accounted for by working memory interference in reinforcement learning computations. These results further our understanding of how multiple systems contribute in parallel to human learning and may have important applications for education and computational psychiatry.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Higher perceptual rating in the current trial was associated with higher expectation rating in the next trial with a different cue type.
“Interestingly, higher perceptual rating in the current trial was associated with higher expectation rating in the next trial with a different cue type (SI Appendix, Figs. S8–S10) which suggests: 1) the existence of a carryover effect (61–63) from the previous to the current trial independent of the cue, or 2) a general learning of stimulus history.”
From Social information creates self-fulfilling prophecies in judgments of pain, vicarious pain, and cognitive effort (2026) · cites this paper as [61]
The Results and Learning phase passages examine how previous correct choices for a given stimulus predict later action choices. Our copy does not report perceptual or expectation ratings, or an association between ratings on successive trials with different cue types (searched: perceptual rating, expectation rating, and cue-independent carryover). (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879 · see citing works
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033 · see citing works
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803 · see citing works
Working Memory: Theories, Models, and Controversies
Baddeley, A. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100422 · Cited by 4,271 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works
Bayesian model selection for group studies
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.03.025 · Cited by 1,423 · see citing works
The Magical Mystery Four: How Is Working Memory Capacity Limited, and Why?
Cowan, N. (2010) · Current Directions in Psychological Science · doi:10.1177/0963721409359277 · Cited by 1,255 · see citing works
Interactive memory systems in the human brain
Poldrack, R. A., Clark, J., Paré-Blagoev, E. J., Shohamy, D., Creso Moyano, J., Myers, C., & Gluck, M. A. (2001) · Nature · doi:10.1038/35107080 · Cited by 1,132 · see citing works
Competition among multiple memory systems: converging evidence from animal and human brain studies
Poldrack, R. A., & Packard, M. G. (2003) · Neuropsychologia · doi:10.1016/s0028-3932(02)00157-4 · Cited by 880 · see citing works
Updating dopamine reward signals
Schultz, W. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.11.012 · Cited by 651 · see citing works
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580 · see citing works
Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making
Collins, A., & Koechlin, E. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001293 · Cited by 558 · cited 1× · see citing works
“…adding undirected noise consists in defining the new mixture choice policy: where U is the uniform random policy (U(a) = 1/nA, with number of actions nA=3), and the parameter 0 < ε < 1 controls the amount of noise (Collins & Frank, 2013; Collins & Koechlin, 2012; Guitart-Masip et al., 2012).”
BackgroundSelf-citationFrank, M. J., & O'Reilly, R. C. (2006) · Behavioral Neuroscience · doi:10.1037/0735-7044.120.3.497 · Cited by 472 · see citing works
Reinforcement learning and episodic memory in humans and animals: an integrative framework
Gershman, S. J., & Daw, N. D. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033625 · Cited by 446 · see citing works
Cognitive control over learning: Creating, clustering, and generalizing task-set structure.
Collins, A. G. E., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0030852 · Cited by 421 · see citing works
Self-citationCollins, A. G. E., & Frank, M. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07980.x · Cited by 419 · see citing works
Self-citationGo and no-go learning in reward and punishment: Interactions between affect and effect
Guitart‐Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 418 · see citing works
Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value
Tai, L.-H., Lee, A. M., Benavidez, N., Bonci, A., & Wilbrecht, L. (2012) · Nature Neuroscience · doi:10.1038/nn.3188 · Cited by 399 · see citing works
A context-based theory of recency and contiguity in free recall.
Sederberg, P. B., Howard, M. W., & Kahana, M. J. (2008) · Psychological Review · doi:10.1037/a0013396 · Cited by 381 · see citing works
Reminders of past choices bias decisions for reward in humans
Bornstein, A. M., Khaw, M. W., Daphna Shohamy, A., & Daw, N. D. (2017) · Nature Communications · doi:10.1038/ncomms15958 · Cited by 222 · see citing works
Dopaminergic control of the striatum for high-level cognition
Cools, R. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.04.002 · Cited by 221 · see citing works
Working Memory Contributions to Reinforcement Learning Impairments in Schizophrenia
Collins, A. G. E., Brown, J. K., Gold, J. M., Waltz, J. A., & Frank, M. J. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0989-14.2014 · Cited by 214 · see citing works
Self-citationReinstated episodic context guides sampling-based decisions for reward
Bornstein, A. M., & Norman, K. A. (2017) · Nature Neuroscience · doi:10.1038/nn.4573 · Cited by 161 · see citing works
Collins, A. G. E., & Frank, M. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720963115 · Cited by 146 · see citing works
Self-citationWorking Memory Load Strengthens Reward Prediction Errors
Collins, A. G. E., Ciullo, B., Frank, M. J., & Badre, D. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2700-16.2017 · Cited by 129 · see citing works
Self-citationEpisodic Memory Encoding Interferes with Reward Learning and Decreases Striatal Prediction Errors
Wimmer, G. E., Braun, E. K., Daw, N. D., & Daphna Shohamy, A. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0204-14.2014 · Cited by 126 · see citing works
Collins, A. G. E., Albrecht, M. A., Waltz, J. A., Gold, J. M., & Frank, M. J. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.05.017 · Cited by 111 · see citing works
Self-citationTaming the beast: extracting generalizable knowledge from computational models of cognition
Nassar, M. R., & Frank, M. J. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.04.003 · Cited by 91 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Burnham, K. P., & Anderson, D. R. (2002). Model Selection and Multi-Model Inference: A Practical Information-Theoretic Approach (Google eBook). Springer. Retrieved from http://www.citeulike.org/group/7954/article/4425594
Cited by
Sorted by citations
Beyond dichotomies in reinforcement learning
Collins, A. G. E., & Cockburn, J. (2020) · Nature reviews. Neuroscience · doi:10.1038/s41583-020-0355-6 · Cited by 128 · see citing works
Self-citationThe interpretation of computational model parameters depends on the context
Eckstein, M. K., Master, S. L., Xia, L., Dahl, R. E., Wilbrecht, L., & Collins, A. G. E. (2022) · eLife · doi:10.7554/elife.75474 · Cited by 116 · see citing works
Self-citationAdvances in modeling learning and decision-making in neuroscience
Collins, A. G. E., & Shenhav, A. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01126-y · Cited by 104 · see citing works
Self-citationDisentangling the systems contributing to changes in learning during adolescence
Master, S. L., Eckstein, M. K., Gotlieb, N., Dahl, R. E., Wilbrecht, L., & Collins, A. G. E. (2019) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2019.100732 · Cited by 98 · see citing works
Self-citationOrigin of perseveration in the trade-off between reward and complexity
Gershman, S. J. (2020) · Cognition · doi:10.1016/j.cognition.2020.104394 · Cited by 97 · see citing works
Representation and computation in visual working memory
Bays, P. M., Schneegans, S., Ma, W. J., & Brady, T. F. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01871-2 · Cited by 111 · see citing works
Yoo, A. H., & Collins, A. G. E. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01808 · Cited by 73 · see citing works
Self-citationFundamental processes in sensorimotor learning: Reasoning, refinement, and retrieval
Tsay, J. S., Kim, H. E., McDougle, S. D., Taylor, J. A., Haith, A. M., Avraham, G., Krakauer, J. W., Collins, A. G. E., & Ivry, R. B. (2024) · eLife · doi:10.7554/elife.91839 · Cited by 80 · see citing works
Self-citationNassar, M. R., Waltz, J. A., Albrecht, M. A., Gold, J. M., & Frank, M. J. (2020) · Brain · doi:10.1093/brain/awaa453 · Cited by 65 · see citing works
Circuits, Networks, and Neuropsychiatric Disease: Transitioning From Anatomy to Imaging
Haber, S. N., Tang, W., Choi, E. Y., Yendiki, A., Liu, H., Jbabdi, S., Versace, A., & Phillips, M. L. (2019) · Biological Psychiatry · doi:10.1016/j.biopsych.2019.10.024 · Cited by 60 · see citing works
The role of executive function in shaping reinforcement learning
Rmus, M., McDougle, S. D., & Collins, A. G. E. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.10.003 · Cited by 61 · see citing works
Self-citationMcDougle, S. D., & Collins, A. G. E. (2020) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01774-z · Cited by 59 · see citing works
Self-citation“Don't [ruminate], be happy”: A cognitive perspective linking depression and anhedonia
Rutherford, A. V., McDougle, S. D., & Joormann, J. (2023) · Clinical Psychology Review · doi:10.1016/j.cpr.2023.102255 · Cited by 57 · see citing works
Revisiting the relationship between the P3b and working memory updating
Rac-Lubashevsky, R., & Kessler, Y. (2019) · Biological Psychology · doi:10.1016/j.biopsycho.2019.107769 · Cited by 56 · see citing works
Policy compression: An information bottleneck in action selection
Lai, L., & Gershman, S. (2021) · The psychology of learning and motivation · doi:10.1016/bs.plm.2021.02.004 · Cited by 77 · see citing works
Long-Term Motor Learning in the “Wild” With High Volume Video Game Data
Listman, J. B., Tsay, J. S., Kim, H. E., Mackey, W. E., & Heeger, D. J. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.777779 · Cited by 45 · see citing works
Humans can navigate complex graph structures acquired during latent learning
Rmus, M., Ritz, H., Hunter, L. E., Bornstein, A. M., & Shenhav, A. (2019) · Cognition · doi:10.1016/j.cognition.2022.105103 · Cited by 40 · see citing works
Uncertainty alters the balance between incremental learning and episodic memory
Nicholas, J., Daw, N. D., & Daphna Shohamy, A. (2022) · eLife · doi:10.7554/elife.81679 · Cited by 38 · see citing works
Human decision making balances reward maximization and policy compression
Lai, L., & Gershman, S. J. (2024) · PLoS Comput. Biol. · doi:10.1371/journal.pcbi.1012057 · Cited by 42 · see citing works
Biomimetic versus arbitrary motor control strategies for bionic hand skill learning
Schone, H. R., Udeozor, M., Moninghoff, M., Rispoli, B., Vandersea, J., Lock, B. A., Hargrove, L. J., Makin, T. R., & Baker, C. I. (2024) · Nature Human Behaviour · doi:10.1038/s41562-023-01811-6 · Cited by 38 · see citing works
Reinforcement learning: bringing together computation and cognition
Collins, A. G. E. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.04.011 · Cited by 30 · see citing works
Self-citationRepresentation and computation in working memoryPreprint
Bays, P. M., Schneegans, S., Ma, W. J., & Brady, T. F. (2022) · doi:10.31234/osf.io/kubr9 · Cited by 31 · see citing works
Event segmentation reveals working memory forgetting rate
Jafarpour, A., Buffalo, E. A., Knight, R. T., & Collins, A. G. E. (2022) · iScience · doi:10.1016/j.isci.2022.103902 · Cited by 31 · see citing works
Self-citationvan den Bosch, R., Hezemans, F. H., Määttä, J. I., Hofmans, L., Papadopetraki, D., Verkes, R., Marquand, A. F., Booij, J., & Cools, R. (2023) · eLife · doi:10.7554/elife.83161 · Cited by 26 · see citing works
Reward influences the allocation but not the availability of resources in visual working memory.
Brissenden, J. A., Adkins, T. J., Hsu, Y. T., & Lee, T. G. (2023) · Journal of Experimental Psychology General · doi:10.1037/xge0001370 · Cited by 25 · see citing works
Impulse control disorder in Parkinson’s disease is associated with abnormal frontal value signalling
Tichelaar, J. G., Sayalı, C., Helmich, R. C. G., & Cools, R. (2023) · Brain · doi:10.1093/brain/awad162 · Cited by 23 · see citing works
Wimmer, G. E., & Poldrack, R. A. (2021) · Memory & Cognition · doi:10.3758/s13421-021-01233-7 · Cited by 24 · see citing works
Dynamic noise estimation: A generalized method for modeling noise fluctuations in decision-making
Li, J., Shi, C., Li, L., & Collins, A. G. E. (2024) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2024.102842 · Cited by 21 · see citing works
Self-citationReflections of Idiographic Long-Term Memory Characteristics In Resting-State Neuroimaging Data
Zhou, P., Sense, F., van Rijn, H., & Stocco, A. (2020) · Cognition · doi:10.1016/j.cognition.2021.104660 · Cited by 22 · see citing works
Russin, J., Pavlick, E., & Frank, M. J. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2510270122 · Cited by 23 · see citing works
Experiential values are underweighted in decisions involving symbolic options
Garcia, B., Lebreton, M. P., Bourgeois‐Gironde, S., & Palminteri, S. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01496-3 · Cited by 19 · see citing works
Rmus, M., He, M., Baribault, B., Walsh, E. G., Festa, E. K., Collins, A. G. E., & Nassar, M. R. (2023) · eLife · doi:10.7554/elife.85243 · Cited by 18 · see citing works
Self-citationExecutive Function Assigns Value to Novel Goal-Congruent Outcomes
McDougle, S. D., Ballard, I. C., Baribault, B., Bishop, S. J., & Collins, A. G. E. (2021) · Cerebral Cortex · doi:10.1093/cercor/bhab205 · Cited by 23 · see citing works
Self-citationThe Reward-Complexity Trade-off in Schizophrenia
Gershman, S. J., & Lai, L. (2021) · Computational Psychiatry · doi:10.5334/cpsy.71 · Cited by 18 · see citing works
Starodub, R., Abella, B. S., Hoyt-Brennan, A. M., Leary, M., Mancini, M. E., Chittams, J. L., & Riegel, B. J. (2020) · International Emergency Nursing · doi:10.1016/j.ienj.2019.100829 · Cited by 16 · see citing works
A causal role of estradiol in human reinforcement learning
Veselič, S., Jocham, G., Gausterer, C., Wagner, B., Ernhoefer-Reßler, M., Lanzenberger, R. R., Eisenegger, C., Lamm, C., & Vermeer, A. B. L. (2021) · Hormones and Behavior · doi:10.1016/j.yhbeh.2021.105022 · Cited by 17 · see citing works
Signatures of contextual interference in implicit sensorimotor adaptation
Tsay, J. S., Irving, C. N., & Ivry, R. B. (2023) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2022.2491 · Cited by 15 · see citing works
Working memory capacity estimates moderate value learning for outcome-irrelevant features
Ben-Artzi, I., Luria, R., & Shahar, N. (2022) · Scientific Reports · doi:10.1038/s41598-022-21832-x · Cited by 11 · see citing works
Weydmann, G. J., Palmieri, I., Simões, R. A. G., Cabral, J. C. C., Eckhardt, J., Tavares, P. D. S., Moro, C. C., Alves, P., Buchmann, S., Schmidt, E., et al. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01982-6 · Cited by 13 · see citing works
Hiser, J., Heilicher, M. O., Botsford, C., Crombie, K. M., Bellani, J., Azar, A., Fonzo, G. A., Nacewicz, B. M., & Cisler, J. M. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104361 · Cited by 16 · see citing works
Veeramani, A., Johnson, A. R., Lee, B. T., & Dowlatshahi, A. S. (2022) · Plastic Surgery · doi:10.1177/22925503221120548 · Cited by 12 · see citing works
Westbrook, A., van den Bosch, R., Hofmans, L., Papadopetraki, D., Määttä, J. I., Collins, A. G. E., Frank, M. J., & Cools, R. (2025) · Nature Communications · doi:10.1038/s41467-025-61099-0 · Cited by 22 · see citing works
Self-citationPost-error Slowing During Instrumental Learning is Shaped by Working Memory-based Choice Strategies
McDougle, S. D. (2021) · Neuroscience · doi:10.1016/j.neuroscience.2021.10.016 · Cited by 11 · see citing works
Individual Differences in Reward‐Based Learning Predict Fluid Reasoning Abilities
Stocco, A., Prat, C. S., & Graham, L. K. (2021) · Cognitive Science · doi:10.1111/cogs.12941 · Cited by 9 · see citing works
Context is key for learning motor skills
Collins, A. G. E., & McDougle, S. D. (2021) · Nature · doi:10.1038/d41586-021-03028-x · Cited by 8 · see citing works
Self-citationChoice Type Impacts Human Reinforcement Learning
Rmus, M., Zou, A. R., & Collins, A. (2022) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01947 · Cited by 10 · see citing works
Self-citationAnhedonia and sensitivity to punishment in schizophrenia, depression and opiate use disorder
Ossola, P., Garrett, N., Biso, L., Bishara, A. J., & Marchesi, C. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.02.120 · Cited by 8 · see citing works
Action chunking as conditional policy compression
Lai, L., Huang, A. Z., & Gershman, S. J. (2022) · doi:10.31234/osf.io/z8yrv · Cited by 9 · see citing works
Hassanzadeh, Z., Bahrami, F., & Dortāj, F. (2024) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2024.1304378 · Cited by 10 · see citing works
Impulsivity Relates to Multi-Trial Choice Strategy in Probabilistic Reversal Learning
Zou, A. R., Lopez, D. E. M., Johnson, S. L., & Collins, A. G. E. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.800290 · Cited by 9 · see citing works
Self-citationA cerebellar population coding model for sensorimotor learningPreprint
Wang, T., & Ivry, R. B. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.07.04.547720 · Cited by 8 · see citing works
A habit and working memory model as an alternative account of human reward-based learning
Collins, A. G. E. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02340-0 · Cited by 7 · see citing works
Self-citationLowered inter-stimulus discriminability hurts incremental contributions to learning
Yoo, A. H., Keglovits, H., & Collins, A. G. E. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01104-5 · Cited by 7 · see citing works
Self-citationSchone, H. R., Udeozor, M., Moninghoff, M., Rispoli, B., Vandersea, J., Lock, B. A., Hargrove, L. J., Makin, T. R., & Baker, C. I. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.02.07.525548 · Cited by 7 · see citing works
Causal learning with interrupted time series data
Zhang, Y., & Rottman, B. M. (2023) · Judgment and Decision Making · doi:10.1017/jdm.2023.29 · Cited by 6 · see citing works
Action chunking as conditional policy compression
Lai, L., Huang, A. Z., & Gershman, S. J. (2025) · Cognition · doi:10.1016/j.cognition.2025.106201 · Cited by 6 · see citing works
Xu, Y., Prat, C. S., Sense, F., van Rijn, H., & Stocco, A. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013485 · Cited by 8 · see citing works
Xu, Y., & Stocco, A. (2021) · Computational Brain & Behavior · doi:10.1007/s42113-021-00102-5 · Cited by 6 · see citing works
How are overweight and obesity associated with reinforcement learning deficits? A systematic review
Weydmann, G. J., Miguel, P. M., Hakim, N., Dubé, L., Silveira, P. P., & Bizarro, L. (2023) · Appetite · doi:10.1016/j.appet.2023.107123 · Cited by 6 · see citing works
Wang, T., & Ivry, R. B. (2025) · Science Advances · doi:10.1126/sciadv.adr4540 · Cited by 4 · see citing works
Anesthesia Resistant Memories in Drosophila, a Working Perspective
Bourouliti, A., & Skoulakis, E. M. C. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms23158527 · Cited by 4 · see citing works
Hallquist, M. N., Hwang, K., Luna, B., & Dombrovski, A. Y. (2024) · Science Advances · doi:10.1126/sciadv.adj2219 · Cited by 4 · see citing works
Haile, T. M., Prat, C. S., & Stocco, A. (2024) · Topics in Cognitive Science · doi:10.1111/tops.12730 · Cited by 4 · see citing works
Frogner, E. R., Dahl, A., Kjelkenes, R., Moberget, T., Collins, A. G. E., Westlye, L. T., & Pedersen, M. L. (2025) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2025.101626 · Cited by 3 · see citing works
Self-citationPark, H., Doh, H., Lee, E., Park, H., & Ahn, W. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011692 · Cited by 3 · see citing works
(2025) · doi:10.1371/journal.pcbi.1012872 · Cited by 3 · see citing works
Self-citationEvent segmentation reveals working memory forgetting ratePreprint
Jafarpour, A., Buffalo, E. A., Knight, R. T., & Collins, A. G. E. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/571380 · Cited by 3 · see citing works
Self-citationRottman, B. M., & Zhang, Y. (2025) · Cognitive Research Principles and Implications · doi:10.1186/s41235-025-00614-9 · Cited by 2 · see citing works
Lim, L. X., Akaishi, R., & Hélie, S. (2025) · Mathematics · doi:10.3390/math13152431 · Cited by 2 · see citing works
Working memory as a representational template for reinforcement learning
Shibata, K., Klar, V. S., Fallon, S. J., Masud Husain, B., & Manohar, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-79119-2 · Cited by 2 · see citing works
Dual effects of dual-tasking on instrumental learning
Ham, H., McDougle, S. D., & Collins, A. G. E. (2025) · Cognition · doi:10.1016/j.cognition.2025.106228 · Cited by 2 · see citing works
Self-citationExecutive function supports single-shot endowment of value to arbitrary transient goalsPreprint
McDougle, S. D., Ballard, I. C., Baribault, B., Bishop, S. J., & Collins, A. G. E. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.10.21.348938 · Cited by 2 · see citing works
Self-citationPlucking a string or playing a G? Choice type impacts human reinforcement learningPreprint
Rmus, M., Zou, A. R., & Collins, A. G. E. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.08.25.457707 · Cited by 2 · see citing works
Self-citationYazdanpanah, A., Jung, H., Soltani, A., & Wager, T. D. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2513856123 · Cited by 1 · see citing works
“S8–S10) which suggests: 1) the existence of a carryover effect (61–63) from the previous to the current trial independent of the cue, or 2) a general learning of stimulus history.”
Neural decision dynamics underlying reinforcement learning and working memory
Pedersen, M. L., Frogner, E. R., Westlye, L. T., & Moberget, T. (2026) · iScience · doi:10.1016/j.isci.2026.115471 · Cited by 1 · see citing works
Stocco, A. (2021) · Journal of Cognition · doi:10.5334/joc.175 · Cited by 1 · see citing works
Stocco, A., Smith, B. M., PeConga, E. K., & Zoellner, L. A. (2021) · doi:10.31234/osf.io/rwz6u · Cited by 1 · see citing works
RL or not RL? Parsing the processes that support human reward-based learning.Preprint
Collins, A. G. E. (2025) · doi:10.31234/osf.io/he3pm_v2 · Cited by 1 · see citing works
Self-citationA Hierarchical Bayesian Implementation of the Experience-Weighted Attraction Model
Zhang, Z., Chandra, S., Kayser, A. S., Hsu, M., & Warren, J. L. (2020) · Computational Psychiatry · doi:10.1162/cpsy_a_00028
Comparing the effects of frontal and temporal neurostimulation on second language learning.
Bice, K., & Prat, C. S. (2020) · Cognitive Science
What is a choice in reinforcement learning
Rmus, M., & Collins, A. G. E. (2020) · eScholarship (California Digital Library) · Cited by 1 · see citing works
Self-citationThe impassable gap between experiential and symbolic valuesPreprint
Garcia, B., Lebreton, M. P., Bourgeois‐Gironde, S., & Palminteri, S. (2022) · doi:10.31234/osf.io/d59tz
Dual Effects of Dual-Tasking on Instrumental LearningPreprint
Ham, H., McDougle, S. D., & Collins, A. G. E. (2024) · doi:10.31234/osf.io/hxe6p
Self-citationDual Effects of Dual-Tasking on Instrumental LearningPreprint
Ham, H., McDougle, S. D., & Collins, A. G. E. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4801088
Self-citationWhat’s in a name: The role of verbalization in reinforcement learning
Schaaf, J. V., Johansson, A., Visser, I., & Huizenga, H. M. (2024) · Psychonomic Bulletin & Review · doi:10.3758/s13423-024-02506-3
Skill learning in the developing brain: Interactions of control and representation systems
Fandakova, Y., & Wenger, E. (2024) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/bs.plm.2024.07.002
Weydmann, G. J., Palmieri, I., Simões, R. A. G., Buchmann, S., Schmidt, E., Alves, P., & Bizarro, L. (2024) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2024.111173
Temporal Dynamics in instrumental Learning: Integrating Entropy Measures and Computational Models
Hassanzadeh, Z., & Bahrami, F. (2024) · National and International Iranian Conference on Biomedical Engineering (ICBME) · doi:10.1109/icbme64381.2024.10894836
Hitchcock, P., Kim, J., & Frank, M. J. (2025) · doi:10.31234/osf.io/82pyz_v2
Hitchcock, P., Kim, J., & Frank, M. J. (2024) · doi:10.31234/osf.io/82pyz_v1
Estimation-uncertainty affects decisions with and without learning opportunities
Aberg, K. C., Antle, L., & Paz, R. (2025) · Nature Communications · doi:10.1038/s41467-025-61960-2
Building and comparing values from different sources
Garcia, B. (2022) · theses.fr (ABES) · doi:10.70675/ee161594z13f1z47c7z99b3zf21f5415b5ca
Biased yet flexible weighting of perceptual and value information during decision-makingPreprint
Garcia, B., Wyart, V., & Bavelier, D. (2026) · Research Square · doi:10.21203/rs.3.rs-9140831/v1
Cognitive resources predict value learning for outcome-irrelevant featuresPreprint
Luria, R., Shahar, N., & Ben-Artzi, I. (2024) · OSF Preprints (OSF Preprints)
Shang, Z., Wang, Y., Li, M., Jin, F., Yang, L., Li, Z., & Yue, C. (2026) · Journal of Experimental Biology · doi:10.1242/jeb.252383
Collins, A. G. E., & Pan, T.-F. (2026) · Nature Human Behaviour · doi:10.1038/s41562-026-02508-2
Self-citationThe influence of working memory on the relationship between anxiety and reinforcement learning
Pereira, P. A., Núñez-Estupiñán, H. X., Weydmann, G. J., & Bizarro, L. (2026) · Learning and Motivation · doi:10.1016/j.lmot.2026.102346
- Authors
- Collins, A. G. E.
- Affiliations
- University of California, Berkeley (US)
- Corresponding author
- Collins, Anne G. E.
- ORCID
- Collins, Anne G. E.
- Venue
- Journal of Cognitive Neuroscience
- Publisher
- MIT Press
- Published
- 2018-10
- Volume
- 30
- Issue
- 10
- Pages
- 1422-1432
- ISSN
- 0898-929X (ISSN-L)
- 1530-8898
- OpenAlex
- W2781809075
- Cited by
- 97
- References
- 29
- Citations by year
- 2026: 6 · 2025: 14 · 2024: 17 · 2023: 15 · 2022: 13 · 2021: 15 · 2020: 11 · 2019: 6
- FWCI
- 4.76
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by-nc
- Locations
- Journal of Cognitive Neuroscience (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by-nc-sa)
- Funders
- Brown University
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- reinforcement learning, working memory, working memory interference, reward prediction error, associative learning, long-term retention, learning speed, computational models of learning, computational psychiatry
- MeSH terms
- Adolescent, Adult, Association Learning, Computer Simulation, Female, Humans, Male, Memory, Short-Term, Reinforcement, Psychology, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
