Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Background When studying learning, researchers directly observe only the participants' choices, which are often assumed to arise from a unitary learning process. However, a number of separable systems, such as working memory (WM) and reinforcement learning (RL), contribute simultaneously to human learning. Identifying each system's contributions is essential for mapping the neural substrates contributing in parallel to behavior; computational modeling can help design tasks that allow such a separable identification of processes, and infer their contributions in individuals. Methods We present a new experimental protocol that separately identifies the contributions of RL and WM to learning, is sensitive to parametric variations in both, and allows us to investigate whether the processes interact. In experiments 1-2, we test this protocol with healthy young adults (n=29 and n=52). In experiment 3, we use it to investigate learning deficits in medicated individuals with schizophrenia (n=49 patients, n=32 controls). Results Experiments 1-2 established WM and RL contributions to learning, evidenced by parametric modulations of choice by load and delay, and reward history, respectively. It also showed interactions between WM and RL, where RL was enhanced under high WM load. Moreover, we observed a cost of mental effort, controlling for reinforcement history: participants preferred stimuli they encountered under low WM load. Experiment 3 revealed selective deficits in WM contributions and preserved RL value learning in individuals with schizophrenia compared to controls. Conclusions Computational approaches allow us to disentangle contributions of multiple systems to learning and, consequently, further our understanding of psychiatric diseases.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Working memory: Looking back and looking forward
Baddeley, A. (2003) · Nature Reviews Neuroscience · doi:10.1038/nrn1201 · Cited by 5,200
Working Memory: Theories, Models, and Controversies
Baddeley, A. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100422 · Cited by 4,271
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“corticostriatal synaptic plasticity (17,18).”
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Self-citationTraining and plasticity of working memory
Klingberg, T. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.05.002 · Cited by 1,585
The Magical Mystery Four: How Is Working Memory Capacity Limited, and Why?
Cowan, N. (2010) · Current Directions in Psychological Science · doi:10.1177/0963721409359277 · Cited by 1,255 · cited 1×
“Indeed, participants were exposed to a large number of images during the learning phase (78 and 39 in experiments 1 and 2, respectively), far exceeding the capacity of WM (7,8,28).”
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Prefrontal cortex and basal ganglia control access to working memory
McNab, F., & Klingberg, T. (2008) · Nature Neuroscience · doi:10.1038/nn2024 · Cited by 989
Negative Symptoms of Schizophrenia Are Associated with Abnormal Effort-Cost Computations
Gold, J. M., Strauss, G. P., Waltz, J. A., Robinson, B. M., Brown, J. K., & Frank, M. J. (2013) · Biological Psychiatry · doi:10.1016/j.biopsych.2012.12.022 · Cited by 421
Self-citation
Cited by
The Tortoise and the Hare: Interactions between Reinforcement Learning and Working Memory
Collins, A. G. E. (2018) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01238 · Cited by 97
Self-citationAttractor-like dynamics in belief updating in schizophrenia
Adams, R. A., Napier, G., Roiser, J. P., Mathys, C., & Gilleen, J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.3163-17.2018 · Cited by 77
“Of note, two other computational studies demonstrating clear working memory parameter differences between Scz and controls also failed to detect any relationship between those parameters and symptom domains (Collins et al., 2014, 2017).”
ResultBetz, L. T., Brambilla, P., Ilankovic, A., Premkumar, P., Kim, M.-S., Raffard, S., Bayard, S., Hori, H., Lee, K.-U., Lee, S. J., et al. (2019) · Schizophrenia Research · doi:10.1016/j.schres.2018.09.009 · Cited by 32
“Importantly, WM interacts with basic processes of reinforcement learning (RL) in learning paradigms (Collins et al., 2017), making the dopaminergic system – implicated in reward processing (Pizzagalli et al., 2008; Schultz et al., 1997; St Onge and Floresco, 2009) – another factor in IGT…”
BackgroundMoran, E. K., Culbreth, A. J., & Barch, D. M. (2022) · Current Topics in Behavioral Neurosciences · doi:10.1007/7854_2022_321 · Cited by 30
Mikus, N., Lamm, C., & Mathys, C. (2025) · Biological Psychiatry · doi:10.1016/j.biopsych.2024.08.021 · Cited by 16
Metadata
- Authors
- Collins, A. G. E., Albrecht, M. A., Waltz, J. A., Gold, J. M., & Frank, M. J.
- Venue
- Biological Psychiatry
- Publisher
- Elsevier BV
- Published
- 2017-09
- Volume
- 82
- Issue
- 6
- Pages
- 431-439
- ISSN
- 0006-3223 (ISSN-L)
- OpenAlex
- W2617674403
- Cited by
- 112
- References
- 14
- FWCI
- 5.15
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia. https://app.cochat.ai/papers/10.1016/j.biopsych.2017.05.017
- The paper
- Collins, A. G. E., Albrecht, M. A., Waltz, J. A., Gold, J. M., & Frank, M. J. (2017). Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia. Biological Psychiatry. https://doi.org/10.1016/j.biopsych.2017.05.017
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.