A theory of actions and habits: The interaction of rate correlation and contiguity systems in free-operant behavior.
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Contemporary theories of instrumental performance assume that responding can be controlled by 2 behavioral systems, 1 goal-directed that encodes the outcome of an action, and 1 habitual that reinforces the response strength of the same action. Here we present a model of free-operant behavior in which goal-directed control is determined by the correlation between the rates of the action and the outcome whereas the total prediction error generated by contiguous reinforcement by the outcome controls habitual response strength. The outputs of these two systems summate to generate a total response strength. This cooperative model addresses the difference in the behavioral impact of ratio and interval schedules, the transition from goal-directed to habitual control with extended training, the persistence of goal-directed control under choice procedures and following extinction, among other phenomena. In these respects, this dual-system model is unique in its account of free-operant behavior. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541
Self-citationActions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217
Self-citationTonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133
Motivational control of goal-directed action
Dickinson, A., & Balleine, B. (1994) · Animal Learning & Behavior · doi:10.3758/bf03199951 · Cited by 995
Self-citationDolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
A specific role for posterior dorsolateral striatum in human habit learning
Tricomi, E., Balleine, B. W., & O’Doherty, J. P. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06796.x · Cited by 769
Coordination of Actions and Habits in the Medial Prefrontal Cortex of Rats
Killcross, S. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.4.400 · Cited by 748
Instrumental Responding following Reinforcer Devaluation
Adams, C. D., & Dickinson, A. (1981) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748108400816 · Cited by 687
Self-citationNeural computations underlying arbitration between model-based and model-free learning
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642
Gremel, C. M., & Costa, R. M. (2013) · Nature Communications · doi:10.1038/ncomms3264 · Cited by 618
Gillan, C. M., Papmeyer, M., Morein-Zamir, S., Sahakian, B. J., Fineberg, N. A., Robbins, T. W., & de Wit, S. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.10071062 · Cited by 609
Determining the neural substrates of goal-directed learning in the human brain
Valentin, V. V., Dickinson, A., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0564-07.2007 · Cited by 529
Self-citationAmphetamine Exposure Enhances Habit Formation
Nelson, A., & Killcross, S. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4305-05.2006 · Cited by 445
Relations between Pavlovian-instrumental transfer and reinforcer devaluation
Holland, P. C. (2004) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/0097-7403.30.2.104 · Cited by 388
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380
Speed/accuracy trade-off between the habitual and the goal-directed processes
Keramati, M., Dezfouli, A., & Piray, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002055 · Cited by 346
The effect of the instrumental training contingency on susceptibility to reinforcer devaluation
Dickinson, A., Nicholas, D. J., & Adams, C. D. (1983) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748308400912 · Cited by 338
Self-citationHabits, action sequences and reinforcement learning
Dezfouli, A., & Balleine, B. W. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08050.x · Cited by 297
Model-based learning protects against forming habits
Gillan, C. M., Otto, A. R., Phelps, E. A., & Daw, N. D. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0347-6 · Cited by 283
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280
Associative learning and animal cognition
Dickinson, A. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0220 · Cited by 219
Self-citationWhen Does Model-Based Control Pay Off?
Kool, W., Cushman, F. A., & Gershman, S. J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005090 · Cited by 211
Contextual control of instrumental actions and habits.
Thrailkill, E. A., & Bouton, M. E. (2015) · Journal of Experimental Psychology: Animal Learning and Cognition · doi:10.1037/xan0000045 · Cited by 165
Effects of Repeated Cocaine Exposure on Habit Learning and Reversal by N-Acetylcysteine
Corbit, L. H., Chieng, B. C., & Balleine, B. W. (2014) · Neuropsychopharmacology · doi:10.1038/npp.2014.37 · Cited by 134
Instrumental uncertainty as a determinant of behavior under interval schedules of reinforcement
DeRusso, A. L. (2010) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2010.00017 · Cited by 111
Soto, F. A., Gershman, S. J., & Niv, Y. (2014) · Psychological Review · doi:10.1037/a0037018 · Cited by 81
Unexpected food outcomes can return a habit to goal-directed action
Bouton, M. E., Broomer, M. C., Rey, C. N., & Thrailkill, E. A. (2020) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2020.107163 · Cited by 65
Some factors that restore goal-direction to a habitual behavior
Trask, S., Shipman, M. L., Green, J. T., & Bouton, M. E. (2020) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2020.107161 · Cited by 63
The Effect of Ratio and Interval Training on Pavlovian-Instrumental Transfer in Mice
Wiltgen, B. J., Sinclair, C., Lane, C., Barrows, F., Molina, M., & Chabanon-Hicks, C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0048227 · Cited by 28
Cited by
Hogarth, L. (2020) · Neuropsychopharmacology · doi:10.1038/s41386-020-0600-8 · Cited by 281
Optimizing exposure therapy with an inhibitory retrieval approach and the OptEx Nexus
Craske, M. G., Treanor, M., Zbozinek, T. D., & Vervliet, B. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104069 · Cited by 195
Context, attention, and the switch between habit and goal-direction in behavior
Bouton, M. E. (2021) · Learning & Behavior · doi:10.3758/s13420-021-00488-z · Cited by 100
Pool, E. R., Gera, R., Fransen, A., Perez, O. D., Cremer, A., Aleksic, M., Tanwisuth, S., Quail, S., Ceceli, A. O., Manfredi, D. A., et al. (2022) · Learning & Memory · doi:10.1101/lm.053413.121 · Cited by 74
Self-citationLeveraging cognitive neuroscience for making and breaking real-world habits
Buabang, E. K., Donegan, K. R., Rafei, P., & Gillan, C. M. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.10.006 · Cited by 34
Metadata
- Authors
- Perez, O. D., & Dickinson, A.
- Venue
- Psychological Review
- Published
- 2020-11
- Volume
- 127
- Issue
- 6
- Pages
- 945-971
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- OpenAlex
- W3024879808
- Cited by
- 91
- References
- 31
- FWCI
- 6.63
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.