The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
From Creatures of Habit to Goal-Directed Learners: Tracking the Developmental Emergence of Model-Based Reinforcement Learning
Open accessAbstract
Theoretical models distinguish two decision-making strategies that have been formalized in reinforcement-learning theory. A model-based strategy leverages a cognitive model of potential actions and their consequences to make goal-directed choices, whereas a model-free strategy evaluates actions based solely on their reward history. Research in adults has begun to elucidate the psychological mechanisms and neural substrates underlying these learning processes and factors that influence their relative recruitment. However, the developmental trajectory of these evaluative strategies has not been well characterized. In this study, children, adolescents, and adults performed a sequential reinforcement-learning task that enabled estimation of model-based and model-free contributions to choice. Whereas a model-free strategy was apparent in choice behavior across all age groups, a model-based strategy was absent in children, became evident in adolescents, and strengthened in adults. These results suggest that recruitment of model-based valuation systems represents a critical cognitive component underlying the gradual maturation of goal-directed behavior.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
Morbidity and mortality weekly report
(1997) · The American Journal of Medicine · doi:10.1016/s0002-9343(97)89440-5 · Cited by 4,684
The construction of reality in the child.
Piaget, J. (1954) · doi:10.1037/11168-000 · Cited by 4,558
Delay of gratification in children.
Mischel, W., Shoda, Y., & Rodriguez, M. L. (1989) · Science · doi:10.1126/science.2658056 · Cited by 3,397
(2006) · PubMed · Cited by 3,378
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationThe variable nature of cognitive control: a dual mechanisms framework
Braver, T. S. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.12.010 · Cited by 2,482
Youth Risk Behavior Surveillance United States, 2015
Kann, L., McManus, T., Harris, W. A., Shanklin, S. L., Flint, K. H., Hawkins, J., Queen, B., Lowry, R., Olsen, E. O., Chyen, D., et al. (2016) · MMWR. Surveillance Summaries · doi:10.15585/mmwr.ss6506a1 · Cited by 2,036
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Self-citationBalleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775
Hot and Cool Executive Function in Childhood and Adolescence: Development and Plasticity
Zelazo, P. D., & Carlson, S. M. (2012) · Child Development Perspectives · doi:10.1111/j.1750-8606.2012.00246.x · Cited by 1,403
Developing mechanisms of self-regulation
POSNER, M. I., & ROTHBART, M. K. (2000) · Development and Psychopathology · doi:10.1017/s0954579400003096 · Cited by 1,349
Increased prefrontal and parietal activity after training of working memory
Olesen, P. J., Westerberg, H., & Klingberg, T. (2004) · Nature Neuroscience · doi:10.1038/nn1165 · Cited by 1,317
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217
The early development of executive functions
Diamond, A. (2006) · Lifespan CognitionMechanisms of Change · doi:10.1093/acprof:oso/9780195169539.003.0006 · Cited by 758
An age-related dissociation between knowing rules and using them
Zelazo, P. D., Frye, D., & Rapus, T. (1996) · Cognitive Development · doi:10.1016/s0885-2014(96)90027-1 · Cited by 728
Developmental neurobiology of cognitive control and motivational systems.
Somerville, L. H., & Casey, B. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.01.006 · Cited by 634
Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions
Wimmer, G. E., & Shohamy, D. (2012) · Science · doi:10.1126/science.1223252 · Cited by 515
Helping children apply their knowledge to their behavior on a dimension‐switching task
Kirkham, N. Z., Cruess, L., & Diamond, A. (2003) · Developmental Science · doi:10.1111/1467-7687.00300 · Cited by 473
Working-memory capacity protects model-based learning from stress
Otto, A. R., Raio, C. M., Chiang, A., Phelps, E. A., & Daw, N. D. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1312011110 · Cited by 465
Self-citationA Brain-Based Account of the Development of Rule Use in Childhood
Bunge, S. A., & Zelazo, P. D. (2006) · Current Directions in Psychological Science · doi:10.1111/j.0963-7214.2006.00419.x · Cited by 433
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380
Self-citationDeveloping Cognitive Control: Three Key Transitions
Munakata, Y., Snyder, H. R., & Chatham, C. H. (2012) · Current Directions in Psychological Science · doi:10.1177/0963721412436807 · Cited by 365
Otto (2013) · doi:10.1177/0956797612463080 · Cited by 352
Chatham, C. H., Frank, M. J., & Munakata, Y. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0810002106 · Cited by 332
The hippocampal-striatal axis in learning, prediction and goal-directed behavior
Pennartz, C. M. A., Ito, R., Verschure, P. F. M. J., Battaglia, F. P., & Robbins, T. W. (2011) · Trends in Neurosciences · doi:10.1016/j.tins.2011.08.001 · Cited by 287
A unique adolescent response to reward prediction errors
Cohen, J. R., Asarnow, R. F., Sabb, F. W., Bilder, R. M., Bookheimer, S. Y., Knowlton, B. J., & Poldrack, R. A. (2010) · Nature Neuroscience · doi:10.1038/nn.2558 · Cited by 285
van den Bos, W., Cohen, M. X., Kahnt, T., & Crone, E. A. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr198 · Cited by 276
Smittenaar, P., FitzGerald, T. H. B., Romei, V., Wright, N. D., & Dolan, R. J. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.009 · Cited by 244
Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1417219112 · Cited by 233
Banishing the Control Homunculi in Studies of Action Control and Behavior Change
Verbruggen, F., McLaren, I. P. L., & Chambers, C. D. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614526414 · Cited by 214
The neuroscience of adolescent decision-making
Hartley, C. A., & Somerville, L. H. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.09.004 · Cited by 214
Self-citationMechanisms for widespread hippocampal involvement in cognition.
Shohamy, D., & Turk-Browne, N. B. (2013) · Journal of Experimental Psychology: General · doi:10.1037/a0034461 · Cited by 187
Cognitive Control Predicts Use of Model-based Reinforcement Learning
Otto, A. R., Skatova, A., Madlon-Kay, S., & Daw, N. D. (2015) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00709 · Cited by 186
Self-citationWhen Younger Learners Can Be Better (or at Least More Open-Minded) Than Older Ones
Gopnik, A., Griffiths, T. L., & Lucas, C. G. (2015) · Current Directions in Psychological Science · doi:10.1177/0963721414556653 · Cited by 176
Neural and Psychological Maturation of Decision-making in Adolescence and Young Adulthood
Christakou, A., Gershman, S. J., Niv, Y., Simmons, A., Brammer, M. J., & Rubia, K. (2013) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00447 · Cited by 157
Of goals and habits: age-related and individual differences in goal-directed decision-making
Eppinger, B., Walter, M., Heekeren, H. R., & Li, S. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00253 · Cited by 156
A theoretical account of cognitive effects in delay discounting
Kurth-Nelson, Z. L., Bickel, W. K., & Redish, A. D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08058.x · Cited by 107
Experiential reward learning outweighs instruction prior to adulthood
Decker, J. H., Lourenco, F. S., Doll, B. B., & Hartley, C. A. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0332-5 · Cited by 96
Self-citationKlossek, U., Russell, J., & Dickinson, A. (2008) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.137.1.39 · Cited by 80
Cited by
LeDoux, J., & Daw, N. D. (2018) · Nature Reviews Neuroscience · doi:10.1038/nrn.2018.22 · Cited by 400
Self-citationWhen Does Model-Based Control Pay Off?
Kool, W., Cushman, F. A., & Gershman, S. J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005090 · Cited by 211
Humans primarily use model-based inference in the two-stage task
Feher da Silva, C., & Hare, T. A. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0905-y · Cited by 153
Reasoning supports utilitarian resolutions to moral dilemmas across diverse measures.
Patil, I., Zucchelli, M. M., Kool, W., Campbell, S., Fornasier, F., Calò, M., Silani, G., Cikara, M., & Cushman, F. (2021) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000281 · Cited by 133
Adolescent brain development and contextual influences: a decade in review
Galván, A. (2021) · Journal of Research on Adolescence · doi:10.1111/jora.12687 · Cited by 77
Metadata
- Authors
- Decker, J. H., Otto, A. R., Daw, N. D., & Hartley, C. A.
- Venue
- Psychological Science
- Publisher
- SAGE Publications
- Published
- 2016-04-15
- Volume
- 27
- Issue
- 6
- Pages
- 848-858
- ISSN
- 0956-7976 (ISSN-L)
- 1467-9280
- OpenAlex
- W2336319629
- Cited by
- 263
- References
- 41
- FWCI
- 14.69
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.