Slips of Action and Sequential Decisions: A Cross-Validation Study of Tasks Assessing Habitual and Goal-Directed Action Control
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Instrumental learning and decision-making rely on two parallel systems: a goal-directed and a habitual system. In the past decade, several paradigms have been developed to study these systems in animals and humans by means of e.g., overtraining, devaluation procedures and sequential decision-making. These different paradigms are thought to measure the same constructs, but cross-validation has rarely been investigated. In this study we compared two widely used paradigms that assess aspects of goal-directed and habitual behavior. We correlated parameters from a two-step sequential decision-making task that assesses model-based (MB) and model-free (MF) learning with a slips-of-action paradigm that assesses the ability to suppress cue-triggered, learnt responses when the outcome has been devalued and is therefore no longer desirable. MB control during the two-step task showed a very moderately positive correlation with goal-directed devaluation sensitivity, whereas MF control did not show any associations. Interestingly, parameter estimates of MB and goal-directed behavior in the two tasks were positively correlated with higher-order cognitive measures (e.g., visual short-term memory). These cognitive measures seemed to (at least partly) mediate the association between MB control during sequential decision-making and goal-directed behavior after instructed devaluation. This study provides moderate support for a common framework to describe the propensity towards goal-directed behavior as measured with two frequently used tasks. However, we have to caution that the amount of shared variance between the goal-directed and MB system in both tasks was rather low, suggesting that each task does also pick up distinct aspects of goal-directed behavior. Further investigation of the commonalities and differences between the MF and habit systems as measured with these, and other, tasks is needed. Also, a follow-up cross-validation on the neural systems driving these constructs across different paradigms would promote the definition and operationalization of measures of instrumental learning and decision-making in humans.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 1×
“Goal-directed instrumental learning has also been proposed to have a formal counterpart in RL, in a family of algorithms known as ‘‘MB’’ RL (Daw et al., 2005; Rangel et al., 2008; Redish et al., 2008).”
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · cited 1×
“Goal-directed instrumental learning has also been proposed to have a formal counterpart in RL, in a family of algorithms known as ‘‘MB’’ RL (Daw et al., 2005; Rangel et al., 2008; Redish et al., 2008).”
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · cited 2×
“Based on an extensive body of animal research, this ability is thought to rely on the functioning of two parallel systems: a reflexive habitual system and a deliberate goal-directed system (Dickinson, 1985; Balleine and Dickinson, 1998a).”
Wood, W., & Rünger, D. (2016) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033417 · Cited by 1,445 · cited 2×
“Second, an absent association between the tasks on the assessment of the habitual system might also reflect the robustness of the habitual/MF system, forming a predominant default mode of response (Wood and Rünger, 2016).”
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226 · cited 2×
“…is described: animals with lesions to dorsolateral striatum and infralimbic cortex are perpetually goal-directed, even after extensive training, whereas animals with dorsomedial striatal and prelimbic cortical lesions are habitual even after only minimal training (Yin et al., 2004, 2005b, 2006).”
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217
The role of the dorsomedial striatum in instrumental conditioning
Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04218.x · Cited by 1,099 · cited 2×
“…is described: animals with lesions to dorsolateral striatum and infralimbic cortex are perpetually goal-directed, even after extensive training, whereas animals with dorsomedial striatal and prelimbic cortical lesions are habitual even after only minimal training (Yin et al., 2004, 2005b, 2006).”
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002 · cited 1×
“…proposed systems are used in instrumental choices, it is essential to ensure suitable instruments that objectively assess covert sub-processes contributing to the constructs and that are simultaneously straightforward for intuitive analyses and application in patient samples (Huys et al., 2016).”
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
A specific role for posterior dorsolateral striatum in human habit learning
Tricomi, E., Balleine, B. W., & O’Doherty, J. P. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06796.x · Cited by 769 · cited 3×
“…and habitual systems in instrumental choices has been studied in animals by (selective) outcome devaluation procedures (Adams and Dickinson, 1981; Balleine and Dickinson, 1998b), a method that has been adapted to human research (Valentin et al., 2007; Tricomi et al., 2009; Horstmann et al., 2015).”
Coordination of Actions and Habits in the Medial Prefrontal Cortex of Rats
Killcross, S. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.4.400 · Cited by 748
Instrumental Responding following Reinforcer Devaluation
Adams, C. D., & Dickinson, A. (1981) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748108400816 · Cited by 687 · cited 1×
“…of the goal-directed and habitual systems in instrumental choices has been studied in animals by (selective) outcome devaluation procedures (Adams and Dickinson, 1981; Balleine and Dickinson, 1998b), a method that has been adapted to human research (Valentin et al., 2007; Tricomi et al.,…”
Disorders of compulsivity: A common bias towards learning habits
Voon, V, Derbyshire, K., Rück, C., Irvine, M. A., Worbe, Y., Enander, J., Schreiber, L. R. N., Gillan, C., Fineberg, N. A., Sahakian, B. J., et al. (2015) · Molecular Psychiatry · doi:10.1038/mp.2014.44 · Cited by 643 · cited 3×
“There is growing evidence that deficient instrumental decision-making based on the dual-systems theory is implicated in multiple disorders (e.g., Gillan et al., 2011; Sjoerds et al., 2013; Horstmann et al., 2015; Voon et al., 2015b; McKim et al., 2016; Reiter et al., 2016).”
InfluentialNeural computations underlying arbitration between model-based and model-free learning
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642
Gillan, C. M., Papmeyer, M., Morein-Zamir, S., Sahakian, B. J., Fineberg, N. A., Robbins, T. W., & de Wit, S. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.10071062 · Cited by 609 · cited 4×
“…successful selective inhibition on the basis of outcome value should be suggestive of dominant goal-directed control through more complex S-R-O associations, which is mediated by anticipation and evaluation of the consequent outcome (see e.g., Gillan et al., 2011; de Wit et al., 2012a,b).”
InfluentialStress prompts habit behavior in humans
Schwabe, L., & Wolf, O. T. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.0979-09.2009 · Cited by 606 · cited 1×
“…seems to depend on trait factors such as healthy aging (Eppinger et al., 2013; de Wit et al., 2014) and cognitive capacities (Otto et al., 2013; Smittenaar et al., 2013; Schad et al., 2014) or state conditions, including stress (Schwabe and Wolf, 2009; Otto et al., 2013; Radenbach et al., 2015).”
Izquierdo, A., Suda, R. K., & Murray, E. A. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.1921-04.2004 · Cited by 588
YIN, H., KNOWLTON, B., & BALLEINE, B. (2006) · Behavioural Brain Research · doi:10.1016/j.bbr.2005.07.012 · Cited by 532
Determining the neural substrates of goal-directed learning in the human brain
Valentin, V. V., Dickinson, A., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0564-07.2007 · Cited by 529
A unified framework for addiction: Vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0800472x · Cited by 501 · cited 1×
“Goal-directed instrumental learning has also been proposed to have a formal counterpart in RL, in a family of algorithms known as ‘‘MB’’ RL (Daw et al., 2005; Rangel et al., 2008; Redish et al., 2008).”
Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04219.x · Cited by 447 · cited 3×
“…is described: animals with lesions to dorsolateral striatum and infralimbic cortex are perpetually goal-directed, even after extensive training, whereas animals with dorsomedial striatal and prelimbic cortical lesions are habitual even after only minimal training (Yin et al., 2004, 2005b, 2006).”
Huys, Q. J. M., Eshel, N., O'Nions, E., Sheridan, L., Dayan, P., & Roiser, J. P. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002410 · Cited by 439
The role of prelimbic cortex in instrumental conditioning
Corbit, L. H., & Balleine, B. W. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.023 · Cited by 423
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380 · cited 4×
“The ubiquity of the goal-directed system (but not the habit system) has been acknowledged previously (Doll et al., 2012).”
Influentialde Wit, S., Watson, P., Harsay, H. A., Cohen, M. X., van de Vijver, I., & Ridderinkhof, K. R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.1088-12.2012 · Cited by 373 · cited 8×
“…parameters describing MB and MF control from the two-step sequential decision-making task (Daw et al., 2011) with a DSI from the slips-of-action paradigm (de Wit et al., 2012b; Worbe et al., 2015), which measures the relative balance of goal-directed and habitual choices on a gradual scale.”
InfluentialGoal-directed learning and obsessive-compulsive disorder
Gillan, C. M., & Robbins, T. W. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0475 · Cited by 348
Sjoerds, Z., de Wit, S., van den Brink, W., Robbins, T. W., Beekman, A. T. F., Penninx, B. W. J. H., & Veltman, D. J. (2013) · Translational Psychiatry · doi:10.1038/tp.2013.107 · Cited by 312 · cited 2×
“There is growing evidence that deficient instrumental decision-making based on the dual-systems theory is implicated in multiple disorders (e.g., Gillan et al., 2011; Sjoerds et al., 2013; Horstmann et al., 2015; Voon et al., 2015b; McKim et al., 2016; Reiter et al., 2016).”
Self-citationMapping value based planning and extensively trained choice in the human brain
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3068 · Cited by 310
Dopamine enhances model-based over model-free choice behavior
Wunderlich, K., Smittenaar, P., & Dolan, R. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.042 · Cited by 294
Model-based learning protects against forming habits
Gillan, C. M., Otto, A. R., Phelps, E. A., & Daw, N. D. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0347-6 · Cited by 283
Smittenaar, P., FitzGerald, T. H. B., Romei, V., Wright, N. D., & Dolan, R. J. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.009 · Cited by 244 · cited 2×
“…seems to depend on trait factors such as healthy aging (Eppinger et al., 2013; de Wit et al., 2014) and cognitive capacities (Otto et al., 2013; Smittenaar et al., 2013; Schad et al., 2014) or state conditions, including stress (Schwabe and Wolf, 2009; Otto et al., 2013; Radenbach et al.,…”
Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1417219112 · Cited by 233
Self-citationDickinson, A. (1994) · Animal Learning and Cognition · doi:10.1016/b978-0-08-057169-0.50009-7 · Cited by 200 · cited 1×
“Importantly, neither of the paradigms compared in this study assesses full ‘‘end-stage’’ habits, which are typically manifest only after extensive overtraining (Colwill and Rescorla, 1988; Dickinson, 1994, 2015).”
de Wit, S., Corlett, P. R., Aitken, M. R., Dickinson, A., & Fletcher, P. C. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1639-09.2009 · Cited by 200
Morris, L. S., Kundu, P., Dowell, N., Mechelmans, D. J., Favre, P., Irvine, M. A., Robbins, T. W., Daw, N., Bullmore, E. T., Harrison, N. A., et al. (2016) · Cortex · doi:10.1016/j.cortex.2015.11.004 · Cited by 198
Marchette, S. A., Bakker, A., & Shelton, A. L. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3634-11.2011 · Cited by 162
Enhanced habit formation in Gilles de la Tourette syndrome
Delorme, C., Salvador, A., Valabrègue, R., Roze, E., Palminteri, S., Vidailhet, M., de Wit, S., Robbins, T., Hartmann, A., & Worbe, Y. (2016) · Brain · doi:10.1093/brain/awv307 · Cited by 154 · cited 7×
“, 2013), Gille de La Tourette syndrome (Delorme et al., 2016), Parkinson’s Disease (de Wit et al.”
InfluentialHabitual versus goal-directed action control in Parkinson disease
de Wit, S., Barker, R. A., Dickinson, A. D., & Cools, R. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21514 · Cited by 116
Reliance on habits at the expense of goal-directed control following dopamine precursor depletion
de Wit, S., Standing, H. R., DeVito, E. E., Robinson, O. J., Ridderinkhof, K. R., Robbins, T. W., & Sahakian, B. J. (2012) · Psychopharmacology · doi:10.1007/s00213-011-2563-2 · Cited by 105
Reduced model-based decision-making in schizophrenia.
Culbreth, A. J., Westbrook, A., Daw, N. D., Botvinick, M., & Barch, D. M. (2016) · Journal of Abnormal Psychology · doi:10.1037/abn0000164 · Cited by 104
Stimulus-outcome interactions during instrumental discrimination learning by rats and humans.
de Wit, S., Niry, D., Wariyar, R., Aitken, M. R. F., & Dickinson, A. (2007) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/0097-7403.33.1.1 · Cited by 93 · cited 1×
“To this end, we correlated parameters assessing the involvement of the two systems from an instrumental learning task with an instructed devaluation slips-of-action phase (de Wit et al., 2007, 2012b) and a two-step sequential decision-making task (Daw et al., 2011).”
Valence-dependent influence of serotonin depletion on model-based choice strategy
Worbe, Y., Palminteri, S., Savulich, G., Daw, N. D., Fernandez-Egea, E., Robbins, T. W., & Voon, V (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.46 · Cited by 75
Tryptophan Depletion Promotes Habitual over Goal-Directed Control of Appetitive Responding in Humans
Worbe, Y., Savulich, G., de Wit, S., Fernandez-Egea, E., & Robbins, T. W. (2015) · International Journal of Neuropsychopharmacology · doi:10.1093/ijnp/pyv013 · Cited by 64
Cited by
Conflicted between goal-directed and habitual control, an fMRI investigation
Watson, P., van Wingen, G., & de Wit, S. (2018) · eneuro · doi:10.1523/eneuro.0240-18.2018 · Cited by 84
Corticostriatal foundations of habits
Seger, C. A. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.01.006 · Cited by 38
Friedel, E. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00302 · Cited by 29
“Therefore, there is a close association between the theoretical concepts of goal-directed and model-based behavior (Friedel et al., 2014; Gillan et al., 2015; Sjoerds et al., 2016).”
Background
Metadata
- Authors
- Sjoerds, Z., Dietrich, A., Deserno, L., de Wit, S., Villringer, A., Heinze, H.-J., Schlagenhauf, F., & Horstmann, A.
- Venue
- Frontiers in Behavioral Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2016-12-20
- Volume
- 10
- Pages
- 234-234
- ISSN
- 1662-5153 (ISSN-L)
- OpenAlex
- W2557614148
- Cited by
- 52
- References
- 44
- FWCI
- 2.15
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.