The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Habitual control of goal selection in humans
Open accessAbstract
Significance: Human cognition makes widespread use of goal-directed planning. However, exhaustive forward planning for tasks of real-world complexity is prohibitively computationally demanding. Much research aims to find efficient mechanisms for approximate planning. We describe an approach to this problem that exploits the computational efficiency of habit learning to select goal states that are subsequently used in planning. We also provide experimental evidence that humans implement this approach. Our findings illuminate the basis of learning and choice in humans, demonstrate an integration between mechanisms of habitual and planned control, and contribute to the development of computationally tractable planning algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Attention to Action: Willed and Automatic Control of Behavior
Norman, D. A., & Shallice, T. (1986) · Consciousness and Self-Regulation · doi:10.1007/978-1-4757-0629-1_1 · Cited by 4,455
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
The prefontral cortex and cognitive control
Miller, E. K. (2000) · Nature Reviews Neuroscience · doi:10.1038/35036228 · Cited by 2,016
Sfard, A. (1991) · Educational Studies in Mathematics · doi:10.1007/bf00302715 · Cited by 1,852
Habits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · cited 1×
“This proposal is consonant with recent research emphasizing the pervasive role of model-free control in related elements of higher-level cognition (18, 19), including the gating of working memory (20) and the construction of hierarchical task representations (21).”
ResultGoal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
The cultural niche: Why social learning is essential for human adaptation
Boyd, R., Richerson, P. J., & Henrich, J. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1100290108 · Cited by 1,303
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917
‘Utilization behaviour’ and its relation to lesions of the frontal lobes
LHERMITTE, F. (1983) · Brain · doi:10.1093/brain/106.2.237 · Cited by 747
Olsson, A., & Phelps, E. A. (2007) · Nature Neuroscience · doi:10.1038/nn1968 · Cited by 622
Hierarchical models of behavior and prefrontal function
Botvinick, M. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.009 · Cited by 532 · cited 2×
“Indeed, it is widely recognized that humans mentally organize their behavior around hierarchically organized goals and subgoals (3, 14, 15).”
BackgroundHuys, Q. J. M., Eshel, N., O'Nions, E., Sheridan, L., Dayan, P., & Roiser, J. P. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002410 · Cited by 439
Cognitive control over learning: Creating, clustering, and generalizing task-set structure.
Collins, A. G. E., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0030852 · Cited by 426 · cited 2×
“This proposal is consonant with recent research emphasizing the pervasive role of model-free control in related elements of higher-level cognition (18, 19), including the gating of working memory (20) and the construction of hierarchical task representations (21).”
ResultOtto (2013) · doi:10.1177/0956797612463080 · Cited by 352 · cited 2×
“Several lines of convergent evidence support the alignment of these mechanisms with habitual and goal-directed control, including functional neuroimaging (24), transcranial magnetic stimulation (35), and manipulations of cognitive load (5) and stress (36), among others (refs.”
BackgroundSmittenaar, P., FitzGerald, T. H. B., Romei, V., Wright, N. D., & Dolan, R. J. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.009 · Cited by 244 · cited 1×
“Several lines of convergent evidence support the alignment of these mechanisms with habitual and goal-directed control, including functional neuro-imaging (24), transcranial magnetic stimulation (35), and manipulations of cognitive load (5) and stress (36), among others (refs.”
BackgroundRetrospective revaluation in sequential decision making: a tale of two systems
Gershman, S. J., Markman, A. B., & Otto, A. R. (2014) · Journal of Experimental Psychology General · doi:10.1037/a0030844 · Cited by 217 · cited 2×
“Methods of integrating habitual and planned control have received less attention (8 – 10), yet real-world tasks often favor elements of each.”
BackgroundThe algorithmic anatomy of model-based evaluation
Daw, N. D., & Dayan, P. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0478 · Cited by 195
Mannella, F., Gurney, K., & Baldassarre, G. (2013) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2013.00135 · Cited by 166
Pezzulo, G., Rigoli, F., & Chersi, F. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00092 · Cited by 163 · cited 1×
“Several promising avenues of research explore how we accomplish this (7, 46 – 49).”
BackgroundThe Neural Basis of Following Advice
Biele, G., Rieskamp, J., Krugel, L. K., & Heekeren, H. R. (2011) · PLoS Biology · doi:10.1371/journal.pbio.1001089 · Cited by 157
Habits as action sequences: hierarchical action control and changes in outcome value
Dezfouli, A., Lingawi, N. W., & Balleine, B. W. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0482 · Cited by 139 · cited 4×
“9 and 10).”
BackgroundInfluential
Cited by
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
Characterizing a psychiatric symptom dimension related to deficits in goal-directed control
Gillan, C. M., Kosinski, M., Whelan, R., Phelps, E. A., & Daw, N. D. (2016) · eLife · doi:10.7554/elife.11305 · Cited by 586
Reinforcement learning and episodic memory in humans and animals: an integrative framework
Gershman, S. J., & Daw, N. D. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033625 · Cited by 446
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475
Prioritized memory access explains planning and hippocampal replay
Mattar, M. G., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0232-z · Cited by 393
Metadata
- Authors
- Cushman, F., & Morris, A.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2015-10-12
- Volume
- 112
- Issue
- 45
- Pages
- 13817-13822
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2228005960
- Cited by
- 163
- References
- 27
- FWCI
- 7.93
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.