The DOI matches its Crossref record; 65 of the 68 references it cites were found in a registry or an index, and 3 are not checked yet.
Humans primarily use model-based inference in the two-stage task
Open accessPublished version of a preprint
Abstract
Distinct model-free and model-based learning processes are thought to drive both typical and dysfunctional behaviours. Data from two-stage decision tasks have seemingly shown that human behaviour is driven by both processes operating in parallel. However, in this study, we show that more detailed task instructions lead participants to make primarily model-based choices that have little, if any, simple model-free influence. We also demonstrate that behaviour in the two-stage task may falsely appear to be driven by a combination of simple model-free and model-based learning if purely model-based agents form inaccurate models of the task because of misconceptions. Furthermore, we report evidence that many participants do misconceive the task in important ways. Overall, we argue that humans formulate a wide variety of learning models. Consequently, the simple dichotomy of model-free versus model-based learning is inadequate to explain behaviour in the two-stage task and connections between reward learning, habit formation and compulsivity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Stan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754
Statsmodels: Econometric and Statistical Modeling with Python
Seabold, S., & Perktold, J. (2010) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-92bf1922-011 · Cited by 6,968
(2015) · doi:10.1007/s11222-016-9696-4 · Cited by 5,611
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Balleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775
Visual fixations and the computation and comparison of value in simple choice
Krajbich, I., Armel, C., & Rangel, A. (2010) · Nature Neuroscience · doi:10.1038/nn.2635 · Cited by 1,408
Research works printed but not linked: 2
- Miller, K. J., Shenhav, A. & Ludvig, E. A. Habits without values. Psychol. Rev. 126, 292–311 (2019). · DOI · Paper page
- Economides, M., Kurth-Nelson, Z., Lübbert, A., Guitart-Masip, M. & Dolan, R. J. Model-based reasoning in humans becomes automatic with training. PLoS Comput. Biol. 11, e1004463 (2015). · DOI
Cited by
Sorted by citations
Items 101–153 of 153
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.003.0006 · Cited by 1
Marzuki, A. A., & Lim, T. V. (2024) · Neuropsychopharmacology · doi:10.1038/s41386-024-01834-1 · Cited by 1
Wyckmans, F., Chatard, A., Kornreich, C., Gruson, D., Jaafari, N., & Noël, X. (2024) · Addictive Behaviors Reports · doi:10.1016/j.abrep.2024.100574 · Cited by 2
The reward positivity is insensitive to reinforcer devaluation
Shaffer, L. S., Crowder, H. D., Kakalec, P. A., Duong, L. T., McDonald, C. G., & Thompson, J. C. (2025) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-025-01306-z · Cited by 1
Sen, P., Koch, K., Schmitz‐Koep, B., Gürsel, D. A., & Knolle, F. (2025) · NeuroImage · doi:10.1016/j.neuroimage.2025.121358 · Cited by 1
Model-based planning in structured foraging environments
Zalabak, T. R., Bustamante, L. A., & Kool, W. (2025) · Cognition · doi:10.1016/j.cognition.2025.106367 · Cited by 1
Devaluation insensitivity of event related potentials associated with food cues
Sambrook, T. D., Wills, A. J., Hardwick, B., & Goslin, J. (2025) · Appetite · doi:10.1016/j.appet.2025.108390 · Cited by 1
Freedom through understanding: instructed knowledge shapes voluntary action choices
Ota, K., Christofilea, E., Charles, L., Jean Daunizeau, A., & Haggard, P. (2026) · Royal Society Open Science · doi:10.1098/rsos.250845 · Cited by 1
A multi-stage anticipated surprise model with dynamic expectation for economic decision-making
Chan, H. K., & Toyoizumi, T. (2024) · Scientific Reports · doi:10.1038/s41598-023-50529-y · Cited by 4
Crombie, K. M., Cisler, J. M., Aguiar, E. J., MacDonald, H. V., Lim, S.-L., Greenwood, B. N., & Hillard, C. J. (2025) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000003754 · Cited by 1
Held, L. K., Lesage, E., Kool, W., & Braem, S. (2025) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-025-01350-9 · Cited by 3
Tracking the contribution of inductive bias to individualized internal modelsPreprint
Török, B., Nagy, D. G., Kiss, M. M., Janacsek, K., Németh, D., & Orbán, G. (2020) · bioRxiv · doi:10.1101/2020.06.22.163295 · Cited by 2
Is the habit system altered in individuals with obesity? A systematic reviewPreprint
Ciria, L. F., Watson, P., Vadillo, M. A., & Luque, D. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/fhkne · Cited by 2
Beyond Instance-Dependent Noise: Generalizing Noisy Label Modeling to Multiple Labeler DependencePreprint
Dawson, G., & Robi Polikar, A. (2022) · doi:10.36227/techrxiv.21129859.v1 · Cited by 1
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learningPreprint
Venditto, S. J. C., Miller, K. J., Brody, C. D., & Daw, N. D. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.02.28.582617 · Cited by 2
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learningPreprint
Venditto, S. J. C., Miller, K. J., Brody, C. D., & Daw, N. D. (2024) · eLife · doi:10.7554/elife.97612 · Cited by 3
Li, Z., Lu, T., Yan, J., Zhang, X., & Li, Y. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.01.24.634822 · Cited by 1
Recasting adaptation as strategy inferencePreprint
Beaumont, S., Khamassi, M., & Domenech, P. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.03.24.645064 · Cited by 2
Brands, A. M. A., Knauth, K., Mathar, D., Lee, S., Kuzmanovic, B., Tittgemeyer, M., & Peters, J. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.07.18.665478 · Cited by 1
Environmental dynamics shape human learning: change points versus random walks
Foucault, C., Weber, L. A., & Hunt, L. (2025) · doi:10.1101/2025.11.26.690700 · Cited by 2
Beyond Instance-Dependent Noise: Generalizing Noisy Label Modeling to Multiple Labeler DependencePreprint
Dawson, G., & Robi Polikar, A. (2022) · doi:10.36227/techrxiv.21129859
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.003.0008
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.003.0003
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.003.0007
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.003.0002
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.003.0004
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.002.0003
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.002.0004
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.002.0006
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.003.0005
Summerfield, C. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780192843883.003.0001
Ten Tips for Running Behavioral Experiments Using Human ParticipantsPreprint
Konovalov, A. (2023) · doi:10.31234/osf.io/6t2sr
The effects of directed and free self-monitoring on goal-directed and habitual decision-makingPreprint
Hossein, M. M., Ershadmanesh, S., Vahabie, A., & Ahmadabadi, M. N. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/prq42
Zuo, Z., Yang, L., & Li, H. (2023) · JUSTC · doi:10.52396/justc-2023-0132
Weydmann, G. J., Palmieri, I., Simões, R. A. G., Buchmann, S., Schmidt, E., Alves, P., & Bizarro, L. (2024) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2024.111173
Individual Differences in Policy Precision: Links to Suicidal Ideation and Network DynamicsPreprint
Yoon, D., Kim, J., Kim, D. H., Shin, D. W., Bong, S. H., Kim, J., Park, H., Jeong, B., & Jeong, B. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.15.618415
Cerrotti, F., Salvador, A., Hamroun, S., Lebreton, M. P., & Palminteri, S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.04.15.648746
Brydevall, M., Suo, C., Albertella, L., Rotaru, K., O’Brien, T., Yücel, M., & Lee, R. S. C. (2025) · NeuroImage · doi:10.1016/j.neuroimage.2025.121362 · Cited by 1
Cao, Y., & Eckstein, M. K. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.07.31.667923
Deep neural network models of emotion understanding
Thornton, M. A. (2025) · Cognition & Emotion · doi:10.1080/02699931.2025.2543569 · Cited by 1
Better than Maximum Likelihood Estimation of Model-based and Model-free Learning Styles
Yazdani, S., Vahabie, A., Araabi, B. N., & Ahmadabadi, M. N. (2025) · Basic and Clinical Neuroscience Journal · doi:10.32598/bcn.2024.5883.1 · Cited by 2
Computational modeling reveals cognitive processes in simple rodent depression tests
Li, Z., Lu, T., Yan, J., Zhang, X., & Li, Y. (2025) · Cell Reports Methods · doi:10.1016/j.crmeth.2025.101246 · Cited by 2
Information uncertainty influences learning strategy from sequentially delayed rewards
Maulhardt, S., Solway, A., & Charpentier, C. J. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013879
Building Goal-Directed Cognitive GraphsPreprint
Gungi, A., Sepúlveda, P., Aitsahalia, I. F., Blanco-Pozo, M., & Iigaya, K. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.18.706628
Environmental dynamics shape human learning: change points versus random walksPreprint
Foucault, C., Weber, L. A. E., & Hunt, L. T. (2026) · eLife · doi:10.7554/elife.110137.1
Environmental dynamics shape human learning: change points versus random walksPreprint
Foucault, C., Weber, L. A. E., & Hunt, L. T. (2026) · eLife · doi:10.7554/elife.110137
Ruan, Z., Li, Z., Luo, X., Yang, Q., Yuan, Z., & Peng, Z. (2026) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2026.03.007
Author Reply: Why goal-directed cycles still explain most phenomena called emotions
Moors, A. (2026) · Emotion Review · doi:10.1177/17540739261422564
Impact of aging on human decision-making in ecological conditions
Aubert, J. D. S. (2023) · theses.fr (ABES) · doi:10.70675/f7143113zd412z4b5fza432z1032ef6ea209
Cognitive Effort in the Two-Step Task: An Active Inference Drift-Diffusion Model Approach
Garrido-Pérez, Á., Lemoine, V., Pednekar, A., Khaluf, Y., & Simoens, P. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-16955-6_2
Ding, W., Cockburn, J., Simon, J. P., Johri, A., Cho, S. J., Oh, S., Feusner, J. D., Tadayonnejad, R., & O’Doherty, J. P. (2026) · Cell Reports · doi:10.1016/j.celrep.2026.117454 · Cited by 1
Bachman, M. D., HajiHosseini, A., Cho, H., Nemrodov, D., Nestor, A., & Hutcherson, C. A. C. (2026) · NeuroImage · doi:10.1016/j.neuroimage.2026.122230
Brands, A. M. A., Knauth, K., Mathar, D., Lee, S., Kuzmanovic, B., Tittgemeyer, M., & Peters, J. (2026) · Journal of Behavioral Addictions · doi:10.1556/2006.2025.00424 · Cited by 1
Metadata
- Authors
- Feher da Silva, C., & Hare, T. A.
- Affiliations
- University of Zurich (CH)
- Corresponding author
- Hare, Todd A.
- Venue
- Nature Human Behaviour
- Published
- 2020-07-06
- Volume
- 4
- Issue
- 10
- Pages
- 1053-1066
- ISSN
- 2397-3374 (ISSN-L)
- OpenAlex
- W3040440677
- Cited by
- 153
- References
- 64
- Citations by year
- 2026: 13 · 2025: 30 · 2024: 25 · 2023: 26 · 2022: 32 · 2021: 20 · 2020: 6
- FWCI
- 11.30
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- Nature Human Behaviour (publishedVersion)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- Repository@Nottingham (University of Nottingham) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- model-based learning, model-free learning, task instructions, misconceptions, reward learning, habit formation, compulsivity, learning models, decision-making
- MeSH terms
- Adult, Choice Behavior, Goals, Habits, Humans, Models, Psychological, Probability Learning, Reinforcement, Psychology, Reward, Task Performance and Analysis, Thinking, Models, Statistical, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
