Stimulus control of actions and habits: A role for reinforcer predictability and attention in the development of habitual behavior.
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Goal-directed actions are instrumental behaviors whose performance depends on the organism's knowledge of the reinforcing outcome's value. In contrast, habits are instrumental behaviors that are insensitive to the outcome's current value. Although habits in everyday life are typically controlled by stimuli that occasion them, most research has studied habits using free-operant procedures in which no discrete stimuli are present to occasion the response. We therefore studied habit learning when rats were reinforced for lever pressing on a random-interval 30-s schedule in the presence of a discriminative stimulus (S) but not in its absence. In Experiment 1, devaluing the reinforcer with taste aversion conditioning weakened instrumental responding in a 30-s S after 4, 22, and 66 sessions of instrumental training. Even extensive practice thus produced goal-directed action, not habit. Experiments 2 and 3 contrastingly found habit when the duration of S was increased from 30 s to 8 min. Experiment 4 then found habit with the 30-s S when it always contained a reinforcer; goal-directed action was maintained when reinforcers were earned at the same rate but occurred in only 50% of Ss (as in the previous experiments). The results challenge the view that habits are an inevitable consequence of repeated reinforcement (as in the law of effect) and instead suggest that discriminated habits develop when the reinforcer becomes predictable. Under those conditions, organisms may pay less attention to their behavior, much as they pay less attention to signals associated with predicted reinforcers in Pavlovian conditioning. (PsycINFO Database Record (c) 2018 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 4×
“Perhaps consistent with theories of attention and learning (Mackintosh, 1975; Pearce & Hall, 1980), conditions that make the reinforcer predictable may encourage the animal to pay less attention its behavior, thus encouraging the development of habit.”
InfluentialA theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733 · cited 1×
“Perhaps consistent with theories of attention and learning (Mackintosh, 1975; Pearce & Hall, 1980), conditions that make the reinforcer predictable may encourage the animal to pay less attention its behavior, thus encouraging the development of habit.”
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541
Wood, W., & Rünger, D. (2016) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033417 · Cited by 1,445 · cited 2×
“It is often assumed that habits merely develop through Thorndike’s (1911) law of effect (e.g., de Wit & Dickinson, 2009; Wood & Rünger, 2016); that is, with repetition of a “trigger-action-reward” sequence, reinforcement gradually strengthens a direct connection between the trigger (S) and the…”
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226
Instrumental Responding following Reinforcer Devaluation
Adams, C. D., & Dickinson, A. (1981) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748108400816 · Cited by 687 · cited 2×
“…studied under free-operant conditions in which the instrumental response can be made repeatedly without a discriminative stimulus ([S] Adams, 1982; Adams & Dickinson, 1981; Dickinson et al., 1995; Dickinson, Nicholas, & Adams, 1983; Nelson & Killcross, 2006; Thrailkill & Bouton, 2015; Yin,…”
Habitual alcohol seeking: time course and the contribution of subregions of the dorsal striatum
Corbit, L. H., Nie, H., & Janak, P. H. (2012) · Biological Psychiatry · doi:10.1016/j.biopsych.2012.02.024 · Cited by 486
Amphetamine Exposure Enhances Habit Formation
Nelson, A., & Killcross, S. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4305-05.2006 · Cited by 445
Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation
Faure, A., Haberland, U., Condé, F., & Massioui, N. E. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.3894-04.2005 · Cited by 402 · cited 2×
“The two other studies (Faure et al., 2005, 2010) also found mixed evidence of habit with discriminated operant responses.”
Relations between Pavlovian-instrumental transfer and reinforcer devaluation
Holland, P. C. (2004) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/0097-7403.30.2.104 · Cited by 388
The effect of the instrumental training contingency on susceptibility to reinforcer devaluation
Dickinson, A., Nicholas, D. J., & Adams, C. D. (1983) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748308400912 · Cited by 338
Associative learning and animal cognition
Dickinson, A. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0220 · Cited by 219 · cited 1×
“Keywords: stimulus control, habit, goal-directed action, attention Recent theories of instrumental learning have emphasized the idea that instrumental behavior can come in two varieties: goaldirected actions and habits (Daw, Niv, & Dayan, 2005; Dickinson, 1985, 1994, 2012).”
Contextual control of instrumental actions and habits.
Thrailkill, E. A., & Bouton, M. E. (2015) · Journal of Experimental Psychology: Animal Learning and Cognition · doi:10.1037/xan0000045 · Cited by 165 · cited 6×
“…conditions in which the instrumental response can be made repeatedly without a discriminative stimulus ([S] Adams, 1982; Adams & Dickinson, 1981; Dickinson et al., 1995; Dickinson, Nicholas, & Adams, 1983; Nelson & Killcross, 2006; Thrailkill & Bouton, 2015; Yin, Knowlton, & Balleine, 2004).”
InfluentialSelf-citationInstrumental uncertainty as a determinant of behavior under interval schedules of reinforcement
DeRusso, A. L. (2010) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2010.00017 · Cited by 111
Attention and expectation in human predictive learning: The role of uncertainty
Hogarth, L., Dickinson, A., Austin, A., Brown, C., & Duka, T. (2008) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470210701643439 · Cited by 97
Uncertainty and predictiveness determine attention to cues during human associative learning
Beesley, T., Nguyen, K. P., Pearson, D., & Le Pelley, M. E. (2015) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2015.1009919 · Cited by 85 · cited 2×
“…when a CS is reinforced on 100% of its presentations, attention and orienting responses decline, whereas when the CS is reinforced on 50% of its presentations, attention and orienting responses remain directed toward the CS (e.g., Beesley et al., 2015; Hogarth et al., 2008; Kaye & Pearce, 1984).”
Cited by
Behavioral and neurobiological mechanisms of pavlovian and instrumental extinction learning
Bouton, M. E., Maren, S., & McNally, G. P. (2021) · Physiological Reviews · doi:10.1152/physrev.00016.2020 · Cited by 395
Self-citationContext, attention, and the switch between habit and goal-direction in behavior
Bouton, M. E. (2021) · Learning & Behavior · doi:10.3758/s13420-021-00488-z · Cited by 100
“As Thrailkill et al. (2018) had found with stimuli that were five times as long and a reinforcement schedule that was 1/10 as rich, the CRF procedure produced habit, whereas PRF maintained goal-direction.”
BackgroundMethodResultSelf-citationUnexpected food outcomes can return a habit to goal-directed action
Bouton, M. E., Broomer, M. C., Rey, C. N., & Thrailkill, E. A. (2020) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2020.107163 · Cited by 65
“, 1995; Holland, 2004; Thrailkill & Bouton, 2015), perhaps because the reinforcer becomes predictable (Thrailkill et al., 2018).”
BackgroundSelf-citationSome factors that restore goal-direction to a habitual behavior
Trask, S., Shipman, M. L., Green, J. T., & Bouton, M. E. (2020) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2020.107161 · Cited by 63
Self-citationLeveraging cognitive neuroscience for making and breaking real-world habits
Buabang, E. K., Donegan, K. R., Rafei, P., & Gillan, C. M. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.10.006 · Cited by 34
Metadata
- Authors
- Thrailkill, E. A., Trask, S., Vidal, P., Alcalá, J. A., & Bouton, M. E.
- Venue
- Journal of Experimental Psychology: Animal Learning and Cognition
- Published
- 2018-10
- Volume
- 44
- Issue
- 4
- Pages
- 370-384
- ISSN
- 2329-8456 (ISSN-L)
- 2329-8464
- OpenAlex
- W2899870041
- Cited by
- 70
- References
- 17
- FWCI
- 3.73
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.