The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Adaptive integration of habits into depth-limited planning defines a habitual-goal–directed spectrum
Open accessAbstract
Significance: Solving complex tasks often requires estimates of the future consequences of current actions. Estimates could be learned from past experience, but they then risk being out of date, or they could be calculated by a form of planning into the future, a process that is computationally taxing. We show that humans integrate learned estimates into their planning calculations, saving mental effort and time. We also show that increasing time pressure leads to reliance on learned estimates after fewer steps of planning. We suggest a normative rationale for this effect using a computational model. Our results provide a perspective on how the brain combines different decision processes collaboratively to exploit their comparative computational advantages.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationNeural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785 · cited 1×
“The previous, discrete, distinction between goal-directed and habitual decision making has been used to illuminate several psychiatric disorders such as addiction (30) and obsessive-compulsive disorder (31, 32).”
BackgroundA Survey of Monte Carlo Tree Search Methods
Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970 · cited 1×
“…leaves for future work richer possibilities, including different depths of planning in different parts of the tree, or indeed using other heuristic value-estimation methods other than model-free learning, such as a rollout mechanism as in Monte Carlo tree search (34) or using social advices.”
BackgroundDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationAn opportunity cost model of subjective effort and task performance
Kurzban, R., Duckworth, A., Kable, J. W., & Myers, J. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12003196 · Cited by 1,381
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Self-citationThe role of the dorsomedial striatum in instrumental conditioning
Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04218.x · Cited by 1,099
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Self-citation
Cited by
What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior
Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.002 · Cited by 1,055
Lieder, F., & Griffiths, T. L. (2020) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1900061x · Cited by 892
Learning task-state representations
Niv, Y. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0470-8 · Cited by 356
Prioritized memory access explains planning and hippocampal replay
Mattar, M. G., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0232-z · Cited by 393
Predictive representations can link model-based reinforcement learning to model-free mechanisms
Russek, E. M., Momennejad, I., Botvinick, M. M., Gershman, S. J., & Daw, N. D. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005768 · Cited by 326
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280
Space, Time, and Fear: Survival Computations along Defensive Circuits
Mobbs, D., Headley, D. B., Ding, W., & Dayan, P. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.12.016 · Cited by 271
Self-citationDeep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269
Cushman, F. (2020) · Behavioral and Brain Sciences · doi:10.1017/s0140525x19001730 · Cited by 260
Anxiety, Depression, and Decision Making: A Computational Perspective
Bishop, S. J., & Gagne, C. (2018) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-080317-062007 · Cited by 240
Planning and navigation as active inference
Kaplan, R., & Friston, K. J. (2018) · Biological Cybernetics · doi:10.1007/s00422-018-0753-2 · Cited by 203
Time-dependent competition between goal-directed and habitual response preparation
Hardwick, R. M., Forrence, A. D., Krakauer, J. W., & Haith, A. M. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0725-0 · Cited by 211
Am i self-conscious? (or does self-organization entail self-consciousness?)
Friston, K. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00579 · Cited by 197
The successor representation: Its computational logic and neural substrates
Gershman, S. J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.0151-18.2018 · Cited by 191
“Second,theSRisunlikelytobeaself-containedreinforcement learningsystem;empirical(Momennejadetal.,2017) Anotherpossibilityis thatthemodel-basedsystemplanstosomedepthandthenuses theSRtocomputeaheuristicvaluefunction(Keramatietal., 2016).”
Forming global estimates of self-performance from local confidence
Rouault, M., Dayan, P., & Fleming, S. M. (2019) · Nature Communications · doi:10.1038/s41467-019-09075-3 · Cited by 134
Self-citationRational use of cognitive resources in human planning
Callaway, F., van Opheusden, B., Gul, S., Das, P., Krueger, P. M., Griffiths, T. L., & Lieder, F. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01332-8 · Cited by 121
“In these cases, the standard assumption that people can search only in the forward direction 2,3,5,7 may be appropriate.”
BackgroundCognitive prostheses for goal achievement
Lieder, F., Chen, O. X., Krueger, P. M., & Griffiths, T. L. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0672-9 · Cited by 32
Recent developments in the habit hypothesis of OCD and compulsive disorders
Gillan, C. M. (2021) · Current Topics in Behavioral Neurosciences · doi:10.1007/7854_2020_199 · Cited by 27
Using computational models of learning to advance cognitive behavioral therapy
Berwian, I. M., Hitchock, P., Pisupati, S., Schoen, G., & Niv, Y. (2025) · Communications Psychology · doi:10.1038/s44271-025-00251-4 · Cited by 16
Metadata
- Authors
- Keramati, M., Smittenaar, P., Dolan, R. J., & Dayan, P.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2016-10-24
- Volume
- 113
- Issue
- 45
- Pages
- 12868-12873
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2535161073
- Cited by
- 203
- References
- 21
- FWCI
- 10.77
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.