The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Reinforcement learning across development: What insights can we draw from a decade of research?
Open accessAbstract
The past decade has seen the emergence of the use of reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety of tasks and model variants, have reached convergent conclusions. In this review, we examine whether the tuning of model parameters that govern different aspects of learning and decision-making processes vary consistently as a function of age, and what neurocognitive developmental changes may account for differences in these parameter estimates across development. We explore whether patterns of developmental change in these estimates are better described by differences in the extent to which individuals adapt their learning processes to the statistics of different environments, or by more static learning biases that emerge across varied contexts. We focus specifically on learning rates and inverse temperature parameter estimates, and find evidence that from childhood to adulthood, individuals become better at optimally weighting recent outcomes during learning across diverse contexts and less exploratory in their value-based decision-making. We provide recommendations for how these two possibilities - and potential alternative accounts - can be tested more directly to build a cohesive body of research that yields greater insight into the development of core learning processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Bartra, O., McGuire, J. T., & Kable, J. W. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.02.063 · Cited by 1,927
Scanning the horizon: towards transparent and reproducible neuroimaging research
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.167 · Cited by 1,410
Galvan, A., Hare, T. A., Parra, C. E., Penn, J., Voss, H., Glover, G., & Casey, B. J. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.1062-06.2006 · Cited by 1,282
Brain systems mediating aversive conditioning: an event-related fMRI study
Büchel, C., Morris, J., Dolan, R. J., & Friston, K. J. (1998) · Neuron · doi:10.1016/s0896-6273(00)80476-6 · Cited by 951
Associative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925
Rational regulation of learning dynamics by pupil-linked arousal systems
Nassar, M. R., Rumsey, K. M., Wilson, R. C., Parikh, K., Heasly, B., & Gold, J. I. (2012) · Nature Neuroscience · doi:10.1038/nn.3130 · Cited by 722
MacDonald, S. W. S., Nyberg, L., & Bäckman, L. (2006) · Trends in Neurosciences · doi:10.1016/j.tins.2006.06.011 · Cited by 664
Braams, B. R., van Duijvenvoorde, A. C. K., Peper, J. S., & Crone, E. A. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.4764-14.2015 · Cited by 617
O'Doherty, J., Critchley, H., Deichmann, R., & Dolan, R. J. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-21-07931.2003 · Cited by 611
Humans use directed and random exploration to solve the explore–exploit dilemma.
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0038199 · Cited by 580
Adolescent risky decision-making: Neurocognitive development of reward and control regions
Van Leijenhorst, L., Moor, B. G., Op de Macks, Z. A., Rombouts, S. A. R. B., Westenberg, P. M., & Crone, E. A. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.02.038 · Cited by 467
Increased Brain Signal Variability Accompanies Lower Behavioral Variability in Development
McIntosh, A. R., Kovacevic, N., & Itier, R. J. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000106 · Cited by 431
Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain
Niv, Y., Edlund, J. A., Dayan, P., & O'Doherty, J. P. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5498-10.2012 · Cited by 356
A unique adolescent response to reward prediction errors
Cohen, J. R., Asarnow, R. F., Sabb, F. W., Bilder, R. M., Bookheimer, S. Y., Knowlton, B. J., & Poldrack, R. A. (2010) · Nature Neuroscience · doi:10.1038/nn.2558 · Cited by 285
Silverman, M. H., Jedd, K., & Luciana, M. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.07.083 · Cited by 277
Homeostatic reinforcement learning for integrating reward collection and physiological stability
Keramati, M., & Gutkin, B. (2014) · eLife · doi:10.7554/elife.04811 · Cited by 218
Hauser, T. U., Iannaccone, R., Walitza, S., Brandeis, D., & Brem, S. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2014.09.018 · Cited by 203
Anticipation of emotionally aversive visual stimuli activates right insula
Simmons, A., Matthews, S. C., Stein, M. B., & Paulus, M. P. (2004) · NeuroReport · doi:10.1097/00001756-200410050-00024 · Cited by 195
Neural Components Underlying Behavioral Flexibility in Human Reversal Learning
Ghahremani, D. G., Monterosso, J., Jentsch, J. D., Bilder, R. M., & Poldrack, R. A. (2010) · Cerebral Cortex · doi:10.1093/cercor/bhp247 · Cited by 180
An adaptive optimal controller for discrete-time Markov environments
Witten, I. H. (1977) · Information and Control · doi:10.1016/s0019-9958(77)90354-0 · Cited by 180
Discovering latent causes in reinforcement learning
Gershman, S. J., Norman, K. A., & Niv, Y. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.07.007 · Cited by 151
Rational variability in children's causal inferences: the sampling hypothesis
Denison, S., Bonawitz, E., Gopnik, A., & Griffiths, T. L. (2013) · Cognition · doi:10.1016/j.cognition.2012.10.010 · Cited by 126
Enhanced striatal sensitivity to aversive reinforcement in adolescents versus adults
Galván, A., & McGlennen, K. M. (2013) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00326 · Cited by 111
Cited by
Childhood as a solution to explore–exploit tensions
Gopnik, A. (2020) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2019.0502 · Cited by 319
Adolescent brain development and contextual influences: a decade in review
Galván, A. (2021) · Journal of Research on Adolescence · doi:10.1111/jora.12687 · Cited by 77
Curiosity: primate neural circuits for novelty and information seeking
Monosov, I. E. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00784-9 · Cited by 44
Enhancing the psychometric properties of the Iowa gambling task using full generative modeling
Sullivan-Toole, H., Haines, N., Dale, K., & Olino, T. M. (2022) · Computational Psychiatry · doi:10.5334/cpsy.89 · Cited by 35
Eckstein, M. K., Summerfield, C., Daw, N. D., & Miller, K. J. (2023) · doi:10.1101/2023.05.17.541226 · Cited by 18
Metadata
- Authors
- Nussenbaum, K., & Hartley, C. A.
- Venue
- Developmental Cognitive Neuroscience
- Publisher
- Elsevier BV
- Published
- 2019-12
- Volume
- 40
- Pages
- 100733
- ISSN
- 1878-9293 (ISSN-L)
- OpenAlex
- W2987754832
- Cited by
- 217
- References
- 26
- FWCI
- 10.55
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.