Humans primarily use model-based inference in the two-stage task
Open accessPublished version of a preprint
Abstract
Distinct model-free and model-based learning processes are thought to drive both typical and dysfunctional behaviours. Data from two-stage decision tasks have seemingly shown that human behaviour is driven by both processes operating in parallel. However, in this study, we show that more detailed task instructions lead participants to make primarily model-based choices that have little, if any, simple model-free influence. We also demonstrate that behaviour in the two-stage task may falsely appear to be driven by a combination of simple model-free and model-based learning if purely model-based agents form inaccurate models of the task because of misconceptions. Furthermore, we report evidence that many participants do misconceive the task in important ways. Overall, we argue that humans formulate a wide variety of learning models. Consequently, the simple dichotomy of model-free versus model-based learning is inadequate to explain behaviour in the two-stage task and connections between reward learning, habit formation and compulsivity.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
<i>Stan</i> : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · see citing works
Statsmodels: Econometric and Statistical Modeling with Python
Seabold, S., & Perktold, J. (2010) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-92bf1922-011 · Cited by 6,673 · see citing works
(2015) · doi:10.1007/s11222-016-9696-4 · Cited by 5,611 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · see citing works
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works
Balleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775 · see citing works
Visual fixations and the computation and comparison of value in simple choice
Krajbich, I., Armel, C., & Rangel, A. (2010) · Nature Neuroscience · doi:10.1038/nn.2635 · Cited by 1,408 · see citing works
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · see citing works
Generating random correlation matrices based on vines and extended onion method
Lewandowski, D., Kurowicka, D., & Joe, H. (2009) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2009.04.008 · Cited by 1,160 · see citing works
Toward a rational and mechanistic account of mental effort
Shenhav, A., Musslick, S., Lieder, F., Kool, W., Griffiths, T. L., Cohen, J. D., & Botvinick, M. M. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031526 · Cited by 1,077 · see citing works
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977 · see citing works
Improving generalization for temporal difference learning: The successor representation
Dayan, P. (1993) · Neural Computation · doi:10.1162/neco.1993.5.4.613 · Cited by 693 · see citing works
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 690 · see citing works
Disorders of compulsivity: A common bias towards learning habits
Voon, V, Derbyshire, K., Rück, C., Irvine, M. A., Worbe, Y., Enander, J., Schreiber, L. R. N., Gillan, C., Fineberg, N. A., Sahakian, B. J., et al. (2015) · Molecular Psychiatry · doi:10.1038/mp.2014.44 · Cited by 643 · see citing works
Characterizing a psychiatric symptom dimension related to deficits in goal-directed control
Gillan, C. M., Kosinski, M., Whelan, R., Phelps, E. A., & Daw, N. D. (2016) · eLife · doi:10.7554/elife.11305 · Cited by 586 · see citing works
Working-memory capacity protects model-based learning from stress
Otto, A. R., Raio, C. M., Chiang, A., Phelps, E. A., & Daw, N. D. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1312011110 · Cited by 472 · see citing works
Redish, A. D. (2016) · Nature reviews. Neuroscience · doi:10.1038/nrn.2015.30 · Cited by 462 · see citing works
The successor representation in human reinforcement learning
Momennejad, I., Russek, E. M., Cheong, J. H., Botvinick, M., Daw, N. D., & Gershman, S. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0180-8 · Cited by 384 · see citing works
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380 · see citing works
Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation
Dayan, P., & Berridge, K. C. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0277-8 · Cited by 357 · see citing works
Otto (2013) · doi:10.1177/0956797612463080 · Cited by 352 · see citing works
Habits, action sequences and reinforcement learning
Dezfouli, A., & Balleine, B. W. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08050.x · Cited by 297 · see citing works
Parsimony, exhaustivity and balanced detection in neocortex
Romagnoni, A., Ribot, J., Bennequin, D., et al. (2016) · INRIA a CCSD electronic archive server · doi:10.1371/journal.pcbi · Cited by 297 · see citing works
Dopamine enhances model-based over model-free choice behavior
Wunderlich, K., Smittenaar, P., & Dolan, R. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.042 · Cited by 294 · see citing works
Model-based choices involve prospective neural activity
Doll, B. B., Duncan, K., Simon, D. A., Shohamy, D., & Daw, N. (2015) · Nature Neuroscience · doi:10.1038/nn.3981 · Cited by 284 · see citing works
Dopamine transients are sufficient and necessary for acquisition of model-based associations
Sharpe, M. J., Chang, C. Y., Liu, M. A., Batchelor, H. M., Mueller, L. E., Jones, J. L., Niv, Y., & Schoenbaum, G. M. (2017) · Nature Neuroscience · doi:10.1038/nn.4538 · Cited by 284 · see citing works
Model-based learning protects against forming habits
Gillan, C. M., Otto, A. R., Phelps, E. A., & Daw, N. D. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0347-6 · Cited by 283 · see citing works
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280 · see citing works
From Creatures of Habit to Goal-Directed Learners
Decker, J. H., Otto, A. R., Daw, N. D., & Hartley, C. A. (2016) · Psychological Science · doi:10.1177/0956797616639301 · Cited by 265 · see citing works
Smittenaar, P., FitzGerald, T. H. B., Romei, V., Wright, N. D., & Dolan, R. J. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.009 · Cited by 244 · see citing works
Dorsal hippocampus contributes to model-based planning
Miller, K. J., Botvinick, M., & Brody, C. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4613 · Cited by 243 · see citing works
Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems
Kool, W., Gershman, S. J., & Cushman, F. A. (2017) · Psychological Science · doi:10.1177/0956797617708288 · Cited by 240 · see citing works
Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1417219112 · Cited by 233 · see citing works
Dezfouli, A., & Balleine, B. W. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003364 · Cited by 221 · see citing works
Retrospective revaluation in sequential decision making: a tale of two systems
Gershman, S. J., Markman, A. B., & Otto, A. R. (2014) · Journal of Experimental Psychology General · doi:10.1037/a0030844 · Cited by 217 · see citing works
When Does Model-Based Control Pay Off?
Kool, W., Cushman, F. A., & Gershman, S. J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005090 · Cited by 211 · see citing works
Cognitive Control Predicts Use of Model-based Reinforcement Learning
Otto, A. R., Skatova, A., Madlon‐Kay, S., & Daw, N. D. (2014) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00709 · Cited by 186 · see citing works
Model-Based and Model-Free Decisions in Alcohol Dependence
Sebold, M., Deserno, L., Nebe, S., Schad, D. J., Garbusow, M., Hägele, C., Keller, J., Jünger, E., Kathmann, N., Smolka, M. N., et al. (2014) · Neuropsychobiology · doi:10.1159/000362840 · Cited by 182 · see citing works
A Pallidus-Habenula-Dopamine Pathway Signals Inferred Stimulus Values
Bromberg-Martin, E. S., Matsumoto, M., Hong, S., & Hikosaka, O. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00158.2010 · Cited by 182 · see citing works
Habitual control of goal selection in humans
Cushman, F. A., & Morris, A. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1506367112 · Cited by 165 · see citing works
Akam, T. E., Costa, R. M., & Dayan, P. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004648 · Cited by 150 · see citing works
Shahar, N., Hauser, T. U., Moutoussis, M., Moran, R., Keramati, M., & Dolan, R. J. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006803 · Cited by 144 · see citing works
Habits as action sequences: hierarchical action control and changes in outcome value
Dezfouli, A., Lingawi, N. W., & Balleine, B. W. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0482 · Cited by 139 · see citing works
Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework
Sadacca, B. F., Jones, J. L., & Schoenbaum, G. M. (2016) · eLife · doi:10.7554/elife.13665 · Cited by 137 · see citing works
Of goals and habits: age-related and individual differences in goal-directed decision-making
Eppinger, B., Walter, M., Heekeren, H. R., & Li, S. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00253 · Cited by 131 · see citing works
Holistic Reinforcement Learning: The Role of Structure and Attention
Radulescu, A., Niv, Y., & Ballard, I. C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.01.010 · Cited by 103 · see citing works
Multiple memory systems as substrates for multiple decision systems
Doll, B. B., Daphna Shohamy, A., & Daw, N. D. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.04.014 · Cited by 92 · see citing works
Axiomatic methods, dopamine and reward prediction error
Caplin, A., & Dean, M. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.007 · Cited by 88 · see citing works
Schad, D. J., Obst, E., Sebold, M., Garbusow, M., Bernhardt, N. R., Javadi, A., Zimmermann, U. S., Smolka, M. N., Heinz, A., Rapp, M. A., et al. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01450 · Cited by 87 · see citing works
Konovalov, A., & Krajbich, I. M. (2016) · Nature Communications · doi:10.1038/ncomms12438 · Cited by 86 · see citing works
Model-Based Reasoning in Humans Becomes Automatic with Training
Economides, M., Kurth-Nelson, Z. L., Lübbert, A., Guitart‐Masip, M., & Dolan, R. J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004463 · Cited by 72 · see citing works
The successor representation in human reinforcement learningPreprint
Momennejad, I., Russek, E. M., Cheong, J., et al. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/083824 · Cited by 69 · see citing works
Addiction as vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. C. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x08004986 · Cited by 68 · see citing works
Shahar, N., Moran, R., Hauser, T. U., Kievit, R., McNamee, D., Moutoussis, M., Dolan, R. J., Bullmore, E. T., Dolan, R. J., Goodyer, I. M., et al. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821647116 · Cited by 66 · see citing works
Planning Complexity Registers as a Cost in Metacontrol
Kool, W., Gershman, S. J., & Cushman, F. A. (2018) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01263 · Cited by 57 · see citing works
Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0258-2 · Cited by 55 · see citing works
Hierarchical control of goal-directed action in the cortical–basal ganglia network
Balleine, B. W., Dezfouli, A., Ito, M., & Doya, K. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.06.001 · Cited by 52 · see citing works
Habits and goals: a motivational perspective on action control
Ceceli, A. O., & Tricomi, E. M. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.12.005 · Cited by 32 · see citing works
Toyama, A., Katahira, K., & Ohira, H. (2019) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2019.03.007 · Cited by 29 · see citing works
Model-Free RL or Action Sequences?
Morris, A., & Cushman, F. A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02892 · Cited by 25 · see citing works
da Silva, C. F., & Hare, T. A. (2018) · PLoS ONE · doi:10.1371/journal.pone.0195328 · Cited by 20 · see citing works
Self-citationHumans are primarily model-based learners in the two-stage taskPreprint
da Silva, C. F., & Hare, T. A. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/682922 · Cited by 15 · see citing works
Self-citationCan model-free reinforcement learning operate over information stored in working-memory?Preprint
da Silva, C. F., Yao, Y.-W., & Hare, T. A. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/107698 · Cited by 4 · see citing works
Self-citationda Silva, C. F., Yao, Y., & Hare, T. A. (2017) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-148191 · Cited by 4 · see citing works
Self-citation(2022) · doi:10.1162/jocn
Flagged first, then by citations
Research works printed but not linked: 5
- 11 Otto, A. R., Gershman, S. J., Markman, A. B. & Daw, N. D. The Curse of Planning. Psychological Science 24 , 751-761 (2013). URL http://journals.sagepub.com/doi/10.1177/ 0956797612463080 .
- 20 Decker, J. H., Otto, A. R., Daw, N. D. & Hartley, C. A. From Creatures of Habit to GoalDirected Learners: Tracking the Developmental Emergence of Model-Based Reinforcement Learning. Psychological Science 27 , 848-858 (2016). URL http://pss.sagepub.com/lookup/doi/10.1177/ 0956797616639301 .
- 51 Miller, K. J., Shenhav, A. & Ludvig, E. A. Habits without values. Psychological Review 126 , 292-311 (2019). URL http://doi.apa.org/getdoi.cfm?doi=10.1037/rev0000120 . · DOI · Paper page
- 56 Economides, M., Kurth-Nelson, Z., Lübbert, A., Guitart-Masip, M. & Dolan, R. J. ModelBased Reasoning in Humans Becomes Automatic with Training. PLOS Computational Biology 11 , e1004463 (2015). URL https://journals.plos.org/ploscompbiol/article?id=10.1371/ journal.pcbi.1004463 .
- 66 Vehtari, A., Gelman, A. & Gabry, J. Practical Bayesian model evaluation using leave-one-out crossvalidation and WAIC. Statistics and Computing (2016). URL http://link.springer.com/10. 1007/s11222-016-9696-4 .
Cited by
Sorted by citations
Using cognitive psychology to understand GPT-3
Binz, M., & Schulz, E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2218523120 · Cited by 559 · see citing works
Reinforcement-learning in fronto-striatal circuits
Averbeck, B. B., & O’Doherty, J. P. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01108-0 · Cited by 127 · see citing works
A foundation model to predict and capture human cognition
Binz, M., Akata, E., Bethge, M., Brändle, F., Callaway, F., Coda-Forno, J., Dayan, P., Demircan, C., Eckstein, M. K., Éltető, N., et al. (2025) · Nature · doi:10.1038/s41586-025-09215-4 · Cited by 119 · see citing works
The interpretation of computational model parameters depends on the context
Eckstein, M. K., Master, S. L., Xia, L., Dahl, R. E., Wilbrecht, L., & Collins, A. G. E. (2022) · eLife · doi:10.7554/elife.75474 · Cited by 116 · see citing works
Vandaele, Y., & Ahmed, S. H. (2020) · Neuropsychopharmacology · doi:10.1038/s41386-020-00899-y · Cited by 91 · see citing works
Functional neuroimaging in psychiatry and the case for failing better
Nour, M. M., Liu, Y., & Dolan, R. J. (2022) · Neuron · doi:10.1016/j.neuron.2022.07.005 · Cited by 87 · see citing works
Decision-making ability, psychopathology, and brain connectivity
Moutoussis, M., Garzón, B., Neufeld, S., Bach, D. R., Rigoli, F., Goodyer, I. M., Bullmore, E. T., Fonagy, P., Jones, P. B., Hauser, T. U., et al. (2021) · Neuron · doi:10.1016/j.neuron.2021.04.019 · Cited by 78 · see citing works
Why and how the brain weights contributions from a mixture of experts
O’Doherty, J. P., Lee, S. W., Tadayonnejad, R., Cockburn, J., Iigaya, K., & Charpentier, C. J. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.10.022 · Cited by 67 · see citing works
Discovering cognitive strategies with tiny recurrent neural networks
Ji-An, L., Benna, M. K., & Mattar, M. G. (2025) · Nature · doi:10.1038/s41586-025-09142-4 · Cited by 64 · see citing works
Model-free decision making is prioritized when learning to avoid harming others
Lockwood, P. L., Klein-Flügge, M. C., Abdurahman, A., & Crockett, M. J. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2010890117 · Cited by 63 · see citing works
The neuroscience of active learning and direct instruction
Dubinsky, J. M., & Hamid, A. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105737 · Cited by 101 · see citing works
A taxonomy of surprise definitions
Modirshanechi, A., Brea, J., & Gerstner, W. (2022) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2022.102712 · Cited by 60 · see citing works
Seow, T. X. F., Benoit, E., Dempsey, C., Jennings, M., Maxwell, A., O'Connell, R., & Gillan, C. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.0031-21.2021 · Cited by 35 · see citing works
Naturalistic reinforcement learning
Wise, T., Emery, K. J., & Radulescu, A. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.08.016 · Cited by 51 · see citing works
Explicit knowledge of task structure is a primary determinant of human model-based action
Castro-Rodrigues, P., Akam, T., Snorasson, I., Camacho, M., Paixão, V., Maia, A., Barahona-Corrêa, B., Dayan, P., Simpson, H. B., Costa, R. M., et al. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01346-2 · Cited by 48 · see citing works
Chen, H., Mojtahedzadeh, N., Belanger, M. J., Nebe, S., Kuitunen‐Paul, S., Sebold, M., Garbusow, M., Huys, Q. J. M., Heinz, A., Rapp, M. A., et al. (2021) · Biological Psychiatry · doi:10.1016/j.biopsych.2021.01.009 · Cited by 42 · see citing works
Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges
Dennison, J. B., Sazhin, D., & Smith, D. V. (2022) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1589 · Cited by 44 · see citing works
Humans can navigate complex graph structures acquired during latent learning
Rmus, M., Ritz, H., Hunter, L. E., Bornstein, A. M., & Shenhav, A. (2019) · Cognition · doi:10.1016/j.cognition.2022.105103 · Cited by 40 · see citing works
Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making
He, X., Modirshanechi, A., Lehmann, M. P., Wulfram Gerstner, B., & Herzog, M. H. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009070 · Cited by 45 · see citing works
Distinct replay signatures for prospective decision-making and memory preservation
Wimmer, G. E., Liu, Y., McNamee, D., & Dolan, R. J. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2205211120 · Cited by 46 · see citing works
Goal-Directed and Habitual Control in Human Substance Use: State of the Art and Future Directions
Doñamayor, N., Ebrahimi, C., Arndt, V. A., Weiß, F., Schlagenhauf, F., & Endraß, T. (2022) · Neuropsychobiology · doi:10.1159/000527663 · Cited by 33 · see citing works
Disrupted state transition learning as a computational marker of compulsivity
Sharp, P. B., Dolan, R., & Eldar, E. (2020) · Psychological Medicine · doi:10.1017/s0033291721003846 · Cited by 30 · see citing works
Rethinking model-based and model-free influences on mental effort and striatal prediction errors
da Silva, C. F., Lombardi, G., Edelson, M. G., & Hare, T. A. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01573-1 · Cited by 31 · see citing works
Self-citationRecent developments in the habit hypothesis of OCD and compulsive disorders
Gillan, C. M. (2021) · Current Topics in Behavioral Neurosciences · doi:10.1007/7854_2020_199 · Cited by 27 · see citing works
Wagner, B., Mathar, D., & Peters, J. (2022) · Computational Psychiatry · doi:10.5334/cpsy.84 · Cited by 24 · see citing works
Valence bias in metacontrol of decision making in adolescents and young adults
Bolenz, F., & Eppinger, B. (2021) · Child Development · doi:10.1111/cdev.13693 · Cited by 26 · see citing works
Buabang, E. K., Boddez, Y., Houwer, J. D., & Moors, A. (2021) · Motivation Science · doi:10.1037/mot0000218 · Cited by 24 · see citing works
Distinct value computations support rapid sequential decisions
Mah, A., Schiereck, S. S., Bossio, V., & Constantinople, C. M. (2023) · Nature Communications · doi:10.1038/s41467-023-43250-x · Cited by 25 · see citing works
Negative symptoms, striatal dopamine and model-free reward decision-making in schizophrenia
Brandl, F., Knolle, F., Avram, M., Leucht, C., Yakushev, I., Priller, J., Leucht, S., Ziegler, S., Wunderlich, K., & Sorg, C. (2023) · Brain · doi:10.1093/brain/awac268 · Cited by 24 · see citing works
Latent-state and model-based learning in PTSD
Cisler, J. M., Dunsmoor, J. E., Fonzo, G. A., & Nemeroff, C. B. (2024) · Trends in Neurosciences · doi:10.1016/j.tins.2023.12.002 · Cited by 25 · see citing works
Shahar, N., Hauser, T., Moran, R., Moutoussis, M., Bullmore, E., Dolan, R., & Nspn consortium (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01642-x · Cited by 14 · see citing works
Summerfield, C. (2022) · doi:10.1093/oso/9780192843883.001.0001 · Cited by 23 · see citing works
Hartogsveld, B., Quaedflieg, C. W. E. M., van Ruitenbeek, P., & Smeets, T. (2021) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2021.105596 · Cited by 21 · see citing works
Grujic, N., Brus, J., Burdakov, D., & Polanía, R. (2022) · Science Advances · doi:10.1126/sciadv.abj8935 · Cited by 20 · see citing works
Moughrabi, N., Botsford, C., Gruichich, T. S., Azar, A., Heilicher, M. O., Hiser, J., Crombie, K. M., Dunsmoor, J. E., Stowe, Z. N., & Cisler, J. M. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119709 · Cited by 19 · see citing works
Hackel, L. M., Kalkstein, D. A., & Mende‐Siedlecki, P. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.004 · Cited by 20 · see citing works
Individual differences in learning positive affective value
Wuensch, L., Pool, E. R., & Sander, D. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.11.001 · Cited by 20 · see citing works
Part-Time Bayesians: Incentives and Behavioral Heterogeneity in Belief Updating
Alós‐Ferrer, C., & Garagnani, M. (2023) · Management Science · doi:10.1287/mnsc.2022.4584 · Cited by 17 · see citing works
Mikuš, N., Korb, S., Massaccesi, C., Gausterer, C., Graf, I., Willeit, M., Eisenegger, C., Lamm, C., Silani, G., & Mathys, C. D. (2022) · eLife · doi:10.7554/elife.79661 · Cited by 14 · see citing works
Hunger improves reinforcement-driven but not planned action
van Swieten, M. M. H., Bogacz, R., & Manohar, S. (2021) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-021-00921-w · Cited by 16 · see citing works
Goal-directed and habitual decision making under stress in gambling disorder: An fMRI study
van Timmeren, T., Piray, P., Goudriaan, A. E., & van Holst, R. J. (2023) · Addictive Behaviors · doi:10.1016/j.addbeh.2023.107628 · Cited by 16 · see citing works
Effects of subclinical depression on prefrontal–striatal model-based and model-free learning
Heo, S., Sung, Y., & Lee, S. W. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009003 · Cited by 16 · see citing works
Nissan, N., Hertz, U., Shahar, N., & Gabay, Y. (2023) · Behavioral and Brain Functions · doi:10.1186/s12993-023-00207-w · Cited by 16 · see citing works
Using smartphones to optimise and scale-up the assessment of model-based planning
Donegan, K. R., Brown, V. M., Price, R. B., Gallagher, E., Pringle, A. J., Hanlon, A. K., & Gillan, C. M. (2023) · Communications Psychology · doi:10.1038/s44271-023-00031-y · Cited by 15 · see citing works
Is the habit system altered in individuals with obesity? A systematic review
Ciria, L. F., Watson, P., Vadillo, M. A., & Luque, D. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.07.006 · Cited by 15 · see citing works
Tracking the contribution of inductive bias to individualised internal models
Török, B., Nagy, D., Kiss, M. M., Janacsek, K., Németh, D., & Orbán, G. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010182 · Cited by 15 · see citing works
Active inference and the two-step task
Gijsen, S., Grundei, M., & Blankenburg, F. (2022) · Scientific Reports · doi:10.1038/s41598-022-21766-4 · Cited by 14 · see citing works
Ruan, Z., Seger, C. A., Yang, Q., Kim, D., Lee, S. W., Chen, Q., & Peng, Z. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1162800 · Cited by 14 · see citing works
Neurocomputational model of compulsivity: deviating from an uncertain goal-directed system
Kim, T., Lee, S. W., Lho, S. K., Moon, S., Kim, M., & Kwon, J. S. (2024) · Brain · doi:10.1093/brain/awae102 · Cited by 14 · see citing works
A practical guide for studying human behavior in the lab
Barbosa, J., Stein, H., Zorowitz, S., Niv, Y., Summerfield, C., Soto‐Faraco, S., & Hyafil, A. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01793-9 · Cited by 13 · see citing works
Spontaneous mind wandering impairs model-based decision making
Liu, S., Rabovsky, M., & Schad, D. J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0279532 · Cited by 13 · see citing works
Neural correlates of model-based behavior in internet gaming disorder and alcohol use disorder
Kwon, M., Choi, H., Park, H., Ahn, W., & Jung, Y. (2024) · Journal of Behavioral Addictions · doi:10.1556/2006.2024.00006 · Cited by 13 · see citing works
Liakoni, V., Lehmann, M. P., Modirshanechi, A., Brea, J., Lutti, A., Gerstner, W., & Preuschoff, K. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118780 · Cited by 14 · see citing works
Impaired belief updating and devaluation in adult women with bulimia nervosa
Berner, L. A., Fiore, V. G., Chen, J. Y., Krueger, A., Kaye, W. H., Viranda, T., & de Wit, S. (2023) · Translational Psychiatry · doi:10.1038/s41398-022-02257-6 · Cited by 13 · see citing works
Hiser, J., Heilicher, M. O., Botsford, C., Crombie, K. M., Bellani, J., Azar, A., Fonzo, G. A., Nacewicz, B. M., & Cisler, J. M. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104361 · Cited by 16 · see citing works
Novelty as a drive of human exploration in complex stochastic environments
Modirshanechi, A., Lin, W.-H., He, X., Herzog, M. H., & Gerstner, W. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2502193122 · Cited by 16 · see citing works
Social Concepts Simplify Complex Reinforcement Learning
Hackel, L. M., & Kalkstein, D. A. (2023) · Psychological Science · doi:10.1177/09567976231180587 · Cited by 11 · see citing works
Understanding dual process cognition via the minimum description length principle
Moskovitz, T., Miller, K. J., Sahani, M., & Botvinick, M. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012383 · Cited by 17 · see citing works
The comparable strategic flexibility of model-free and model-based learningPreprint
Filipowicz, A. L. S., Levine, J., Piasini, E., Tavoni, G., Kable, J. W., & Gold, J. I. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2019.12.28.879965 · Cited by 10 · see citing works
Knolle, F., Sen, P., Culbreth, A., Koch, K., Schmitz-Koep, B., Gürsel, D. A., Wunderlich, K., Avram, M., Berberich, G., Sorg, C., et al. (2024) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.240020 · Cited by 9 · see citing works
Hogarth, L. (2024) · Addiction Biology · doi:10.1111/adb.13435 · Cited by 8 · see citing works
Imbalanced goal-directed and habitual control in individuals with internet gaming disorder
Lei, W., Huang, Y., Peng, Y., Chen, G., Gong, K., Liu, K., Wu, D., Qin, C., Cheng, X., Tian, M., et al. (2025) · Journal of Behavioral Addictions · doi:10.1556/2006.2025.00037 · Cited by 10 · see citing works
Inefficient prioritization of task-relevant attributes during instrumental information demand
Rischall, I., Hunter, L., Jensen, G., & Gottlieb, J. P. (2023) · Nature Communications · doi:10.1038/s41467-023-38821-x · Cited by 9 · see citing works
Colas, J. T., Dundon, N. M., Gerraty, R. T., Saragosa-Harris, N. M., Szymula, K. P., Tanwisuth, K., Tyszka, J. M., van Geen, C., Ju, H., Toga, A. W., et al. (2022) · Human Brain Mapping · doi:10.1002/hbm.25988 · Cited by 8 · see citing works
Bayramoğlu, G., & Öztürk, O. (2024) · Journal of Economic Surveys · doi:10.1111/joes.12676 · Cited by 7 · see citing works
Signatures of heuristic-based directed exploration in two-step sequential decision task behaviour
Brands, A. M. A., Mathar, D., & Peters, J. (2025) · Computational Psychiatry · doi:10.5334/cpsy.101 · Cited by 7 · see citing works
Crombie, K. M., Azar, A., Botsford, C., Heilicher, M. O., Hiser, J., Moughrabi, N., Gruichich, T. S., Schomaker, C. M., & Cisler, J. M. (2023) · Journal of Mood and Anxiety Disorders · doi:10.1016/j.xjmad.2023.100015 · Cited by 6 · see citing works
How are overweight and obesity associated with reinforcement learning deficits? A systematic review
Weydmann, G. J., Miguel, P. M., Hakim, N., Dubé, L., Silveira, P. P., & Bizarro, L. (2023) · Appetite · doi:10.1016/j.appet.2023.107123 · Cited by 6 · see citing works
Defining and Measuring Habits Across Different Fields of Research
Watson, P. (2024) · Springer eBooks · doi:10.1007/978-3-031-55889-4_1 · Cited by 5 · see citing works
Novelty is not Surprise: Human exploratory and adaptive behavior in sequential decision-makingPreprint
He, X., Modirshanechi, A., Lehmann, M. P., Gerstner, W., & Herzog, M. H. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.09.24.311084 · Cited by 5 · see citing works
Does the Habit Theory of Addictions Extend to Disordered Gambling?
van Timmeren, T., & Clark, L. (2026) · Current Addiction Reports · doi:10.1007/s40429-026-00715-5 · Cited by 4 · see citing works
Shorter planning depth and higher response noise during sequential decision-making in old age
Steffen, J., Marković, D., Glöckner, F., Neukam, P. T., Kiebel, S. J., Li, S., & Smolka, M. N. (2023) · Scientific Reports · doi:10.1038/s41598-023-33274-0 · Cited by 4 · see citing works
Yang, L., Jin, F., Yang, L., Li, J., Zhi-hui, L., Li, M., & Shang, Z. (2024) · Animals · doi:10.3390/ani14030431 · Cited by 4 · see citing works
Working Memory Guides Action Valuation in Model-based Decision-making Strategy
Zuo, Z., Yang, L., Wang, H., & Li, H. (2024) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_02237 · Cited by 4 · see citing works
Looking deeper into the algorithms underlying human planning
Kuperwajs, I., Russek, E. M., Mattar, M. G., Ma, W. J., & Griffiths, T. L. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.06.006 · Cited by 4 · see citing works
Validating Habitual and Goal-Directed Decision-Making Performance Online in Healthy Older Adults
Ito, K. L., Cao, L., Reinberg, R., Keller, B., Monterosso, J. R., Schweighofer, N., & Liew, S. (2021) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2021.702810 · Cited by 4 · see citing works
The Component Processes of Complex Planning Follow Distinct Developmental Trajectories
Ma, I., Phaneuf-Hadd, C. V., van Opheusden, B., Ma, W. J., & Hartley, C. A. (2023) · Conference on Cognitive Computational Neuroscience (CCN) · doi:10.32470/ccn.2023.1330-0 · Cited by 4 · see citing works
Contextual overtraining accelerates habit formation in new stimuliPreprint
Lesage, E., & Verguts, T. (2021) · doi:10.31234/osf.io/7m6bh · Cited by 4 · see citing works
Multi-step inference can be improved across the lifespan with individualized memory interventionsPreprint
Noh, S. M., Cooper, K. W., Guo, S., Zhou, D., Stark, C. E. L., & Bornstein, A. M. (2023) · doi:10.31234/osf.io/3mhj6 · Cited by 6 · see citing works
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learningPreprint
Venditto, S. J. C., Miller, K. J., Brody, C. D., & Daw, N. D. (2024) · eLife · doi:10.7554/elife.97612.2 · Cited by 4 · see citing works
Luna, R., Vadillo, M. A., & Luque, D. (2023) · Cortex · doi:10.1016/j.cortex.2023.06.009 · Cited by 3 · see citing works
Noradrenergic and Dopaminergic modulation of meta-cognition and meta-control
Ershadmanesh, S., Rajabi, S., Rostami, R., Moran, R., & Dayan, P. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012675 · Cited by 3 · see citing works
Memory for rewards guides retrieval
Nagel, J., Morgan, D. P., Gürsoy, Ç. N., Sander, S., Kern, S., & Feld, G. B. (2024) · Communications Psychology · doi:10.1038/s44271-024-00074-9 · Cited by 3 · see citing works
Hong, Z. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3524882 · Cited by 3 · see citing works
Evidence for shallow cognitive maps in schizophrenia
Karagoz, A. B., Moran, E. K., Barch, D. M., Kool, W., & Reagh, Z. M. (2025) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-025-01283-3 · Cited by 3 · see citing works
Critical intelligence: Computing defensive behaviour
Brochard, J., Dayan, P., & Bach, D. R. (2025) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2025.106213 · Cited by 3 · see citing works
Disorders of compulsivity: Deficits in arbitrating learning strategies
Ruan, Z., Liu, S., Liu, Y. A., Yang, Q., & Peng, Z. (2024) · Addiction Biology · doi:10.1111/adb.13433 · Cited by 3 · see citing works
Distinct replay signatures for prospective decision-making and memory preservationPreprint
Wimmer, G. E., Liu, Y., McNamee, D., & Dolan, R. J. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.08.467745 · Cited by 3 · see citing works
Inefficient Prioritization of Task-Relevant Attributes During Instrumental Information DemandPreprint
Rischall, I., Hunter, L., Jensen, G., & Gottlieb, J. P. (2022) · doi:10.31234/osf.io/vsqxk · Cited by 3 · see citing works
The construction and use of cognitive maps in model-based controlPreprint
Karagoz, A. B., Reagh, Z. M., & Kool, W. (2022) · doi:10.31234/osf.io/ngqwa · Cited by 3 · see citing works
Hamroun, S., Lebreton, M. P., & Palminteri, S. (2022) · doi:10.31234/osf.io/2uw85 · Cited by 3 · see citing works
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learningPreprint
Venditto, S. J. C., Miller, K. J., Brody, C. D., & Daw, N. D. (2024) · eLife · doi:10.7554/elife.97612.1 · Cited by 3 · see citing works
A computational principle of habit formationPreprint
Lakshminarasimhan, K. J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.12.618033 · Cited by 3 · see citing works
Model-free decision-making underlies motor errors in rapid sequential movements under threat
Sankhe, P., & Haruno, M. (2024) · Communications Psychology · doi:10.1038/s44271-024-00123-3 · Cited by 2 · see citing works
Model-based algorithms shape automatic evaluative processing
Melnikoff, D. E., & Kurdi, B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2417068122 · Cited by 2 · see citing works
Kastner, D. B., Yokota, N. O., Lee, C. Y., Holobetz, C., Chaudhry, A., Williams, G., Ton, J. N., Mehra, M., Kharazia, V., Parkinson, D. Y., et al. (2026) · Nature Communications · doi:10.1038/s41467-026-76681-3 · Cited by 2 · see citing works
Integrating neuro-psychological habit research into consumer choice models
Webb, R., Fong, J., Landry, P., Levine, J., Wellsjo, A., Natan, O., Mazar, A., Zhao, C., Lally, P. J., de Wit, S., et al. (2025) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2025.10.002 · Cited by 2 · see citing works
Ebrahimi, C., Musial, M. P. M., Doñamayor, N., Prychynenko, D. S., Bode, E. L., Spanagel, R., Heinz, A., Deserno, L., Smolka, M. N., Ebner‐Priemer, U., et al. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03660-5 · Cited by 2 · see citing works
A web-based implementation of the two-step human reinforcement learning taskPreprint
Kinley, I. (2023) · doi:10.31234/osf.io/b5x3w · Cited by 2 · see citing works
Maturation of striatal dopamine supports the development of habitual behavior through adolescencePreprint
Petrie, D. J., Parr, A. C., Sydnor, V. J., Ojha, A., Foran, W., Tervo‐Clemmens, B., Calabro, F. J., & Luna, B. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.01.06.631527 · Cited by 2 · see citing works
- Authors
- Feher da Silva, C., & Hare, T. A.
- Affiliations
- University of Zurich (CH)
- Corresponding author
- Hare, Todd A.
- Venue
- Nature Human Behaviour
- Published
- 2020-07-06
- Volume
- 4
- Issue
- 10
- Pages
- 1053-1066
- ISSN
- 2397-3374 (ISSN-L)
- OpenAlex
- W3040440677
- Cited by
- 153
- References
- 64
- Citations by year
- 2026: 13 · 2025: 30 · 2024: 25 · 2023: 26 · 2022: 32 · 2021: 20 · 2020: 6 · 2019: 1
- FWCI
- 11.30
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- Nature Human Behaviour (publishedVersion)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- Repository@Nottingham (University of Nottingham) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- model-based learning, model-free learning, task instructions, misconceptions, reward learning, habit formation, compulsivity, learning models, decision-making
- MeSH terms
- Adult, Choice Behavior, Goals, Habits, Humans, Models, Psychological, Probability Learning, Reinforcement, Psychology, Reward, Task Performance and Analysis, Thinking, Models, Statistical, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
