Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
A well-established notion in cognitive neuroscience proposes that multiple brain systems contribute to choice behaviour. These include: (1) a model-free system that uses values cached from the outcome history of alternative actions, and (2) a model-based system that considers action outcomes and the transition structure of the environment. The widespread use of this distinction, across a range of applications, renders it important to index their distinct influences with high reliability. Here we consider the two-stage task, widely considered as a gold standard measure for the contribution of model-based and model-free systems to human choice. We tested the internal/temporal stability of measures from this task, including those estimated via an established computational model, as well as an extended model using drift-diffusion. Drift-diffusion modeling suggested that both choice in the first stage, and RTs in the second stage, are directly affected by a model-based/free trade-off parameter. Both parameter recovery and the stability of model-based estimates were poor but improved substantially when both choice and RT were used (compared to choice only), and when more trials (than conventionally used in research practice) were included in our analysis. The findings have implications for interpretation of past and future studies based on the use of the two-stage task, as well as for characterising the contribution of model-based processes to choice behaviour.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764 · cited 1×
“However, a decision-making literature indicates RT data is an important source of information [25,26], with a recent study suggesting it might improve parameter identifiability in reinforcement learning paradigms [27].”
BackgroundThe Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks
Ratcliff, R., & McKoon, G. (2008) · Neural Computation · doi:10.1162/neco.2008.12-06-420 · Cited by 3,059 · cited 2×
“Giving the known relationship between discriminability and RT [33,42] this observation allowed us to elaborate a second score based on second-stage RTs (RT2) [4,30].”
BackgroundMethodMonsell, S. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00028-7 · Cited by 2,984
Chunking mechanisms in human learning
GOBET, F., LANE, P., CROKER, S., CHENG, P., JONES, G., OLIVER, I., & PINE, J. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01662-4 · Cited by 1,157
Goal-directed and habitual control in the basal ganglia: implications for Parkinson's disease
Redgrave, P., Rodriguez, M., Smith, Y., Rodriguez-Oroz, M. C., Lehericy, S., Bergman, H., Agid, Y., DeLong, M. R., & Obeso, J. A. (2010) · Nature reviews. Neuroscience · doi:10.1038/nrn2915 · Cited by 1,134 · cited 1×
“Theoretical questions regarding human planning [2], reasoning [3], development [4], voluntary-control [5], learning under uncertainty [6], motor control [7], and deployment of attentional resources [8], all invoke this dichotomy.”
BackgroundDolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Self-citationMontague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809
Self-citationNeural computations underlying arbitration between model-based and model-free learning
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642
Gremel, C. M., & Costa, R. M. (2013) · Nature Communications · doi:10.1038/ncomms3264 · Cited by 618
Characterizing a psychiatric symptom dimension related to deficits in goal-directed control
Gillan, C. M., Kosinski, M., Whelan, R., Phelps, E. A., & Daw, N. D. (2016) · eLife · doi:10.7554/elife.11305 · Cited by 586
Huang, V. S., Haith, A., Mazzoni, P., & Krakauer, J. W. (2011) · Neuron · doi:10.1016/j.neuron.2011.04.012 · Cited by 472
Otto (2013) · doi:10.1177/0956797612463080 · Cited by 352 · cited 1×
“This two-stage decision task is considered a gold-standard for measuring model-based/ model-free contributions to choice behaviour across computational [21], neuroimaging [2], behavioural [22], developmental [4] and clinical studies [13,23].”
Mapping value based planning and extensively trained choice in the human brain
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3068 · Cited by 310
Self-citationDopamine enhances model-based over model-free choice behavior
Wunderlich, K., Smittenaar, P., & Dolan, R. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.042 · Cited by 294
Self-citationModel-based learning protects against forming habits
Gillan, C. M., Otto, A. R., Phelps, E. A., & Daw, N. D. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0347-6 · Cited by 283
Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1417219112 · Cited by 233
Self-citationWhen Does Model-Based Control Pay Off?
Kool, W., Cushman, F. A., & Gershman, S. J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005090 · Cited by 211 · cited 2×
“First-stage choices provide descriptive measures as well as model parameters that quantify the contribution of model-free and model-based systems [21].”
BackgroundReduced model-based decision-making in schizophrenia.
Culbreth, A. J., Westbrook, A., Daw, N. D., Botvinick, M., & Barch, D. M. (2016) · Journal of Abnormal Psychology · doi:10.1037/abn0000164 · Cited by 104
Corticostriatal foundations of habits
Seger, C. A. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.01.006 · Cited by 38 · cited 2×
“Theoretical questions regarding human planning [2], reasoning [3], development [4], voluntary-control [5], learning under uncertainty [6], motor control [7], and deployment of attentional resources [8], all invoke this dichotomy.”
BackgroundMethod
Cited by
Humans primarily use model-based inference in the two-stage task
Feher da Silva, C., & Hare, T. A. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0905-y · Cited by 153
Identifying Transdiagnostic Mechanisms in Mental Health Using Computational Factor Modeling
Wise, T., Robinson, O. J., & Gillan, C. M. (2023) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.09.034 · Cited by 108
“In tandem, there has been renewed focus on the reliability of the tasks we employ (106 – 109) and efforts to harmonize tasks across labs and species (110 – 112).”
Stress-sensitive inference of task controllability
Ligneul, R., Mainen, Z. F., Ly, V., & Cools, R. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01306-w · Cited by 60
“It is known that decision times slow down following state prediction errors 33,34 .”
Raio, C. M., Konova, A. B., & Otto, A. R. (2020) · Scientific Reports · doi:10.1038/s41598-020-64540-0 · Cited by 52
Enhancing the psychometric properties of the Iowa gambling task using full generative modeling
Sullivan-Toole, H., Haines, N., Dale, K., & Olino, T. M. (2022) · Computational Psychiatry · doi:10.5334/cpsy.89 · Cited by 35
“…poor test-retest reliability, as parameters from other RL models, when modeled separately across time points, generally show only modest reliability (e.g., r = .48–.57, Chung, et al., 2017; r = .30, Moutoussis, et al., 2018; r = .63, Price, Brown & Siegle, 2019; r = .16–.20, Shahar, et al., 2019).”
Metadata
- Authors
- Shahar, N., Hauser, T. U., Moutoussis, M., Moran, R., Keramati, M., NSPN consortium, & Dolan, R. J.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-02-13
- Volume
- 15
- Issue
- 2
- Pages
- e1006803
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2914262018
- Cited by
- 154
- References
- 19
- FWCI
- 10.55
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.