The effect of the instrumental training contingency on susceptibility to reinforcer devaluation
Abstract
Two experiments investigated performance of instrumental lever pressing by rats following post-conditioning devaluation of the sucrose reinforcer produced by establishing an aversion to it. In Experiment I rats responded less in an extinction test after being averted from the sucrose following training on a ratio schedule, but not following an equivalent amount of training on an interval schedule. This was true even though the devalued sucrose would not act as an effective reinforcer on either the ratio or interval schedule. Experiment II provided a further investigation of the insensitivity of interval responding to reinforcer devaluation by comparing test performance under simple extinction with responding when the devalued reinforcer was presented on either a response-contingent or non-contingent schedule during the test. Once again simple extinction performance was unaffected by prior reinforcer devaluation. Furthermore, neither non-contingent nor contingent presentations of the devalued reinforcer significantly depressed responding below the level seen in the extinction condition. Ratio, but not interval performance appears to be controlled by knowledge about the instrumental contingency that encodes specific properties of the training reinforcer.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Extended training on interval schedules of reinforcement involving single actions and single outcomes promotes habitual responding.
“Various circumstances promote habitual responding, notably extended training on interval schedules of reinforcement involving single actions and single outcomes (Dickinson and Charnock, 1985; Dickinson and Balleine, 2002; Dickinson et al., 1983).”
From Goals and Habits in the Brain (2013) · cites this paper as Dickinson et al., 1983
Habits develop more rapidly with behaviors reinforced on interval as opposed to ratio schedules.
“habits develop more rapidly with behaviors reinforced on interval as opposed to ratio schedules”
From Context, attention, and the switch between habit and goal-direction in behavior (2021) · cites this paper as Dickinson et al., 1983
Habits appear to develop more readily when responding is reinforced on interval than on ratio reinforcement schedules.
“Habits appear to develop more readily when responding is reinforced on interval than on ratio reinforcement schedules (e.g., Dickinson et al., 1983).”
From Context, attention, and the switch between habit and goal-direction in behavior (2021) · cites this paper as Dickinson et al., 1983
The tight press-reward relationship of the random-ratio schedule encourages action-outcome learning and use of such knowledge to support agency and goal-directed decision making.
“The tight press-reward relationship of this regime encourages action-outcome learning and, together with the short training duration, the use of such knowledge to support agency and goal-directed decision making38.”
From A dual-pathway architecture for stress to disrupt agency and promote habit (2025) · cites this paper as [38]
The looser action-outcome relationship of a random-interval schedule is more permissive for habits than a ratio reinforcement schedule.
“However, the looser action-outcome relationship is more permissive for habits than a ratio reinforcement schedule38,42”
From A dual-pathway architecture for stress to disrupt agency and promote habit (2025) · cites this paper as [38]
An RR schedule of reinforcement tends to promote action-outcome learning and goal-directed decision making.
“We chose an RR schedule of reinforcement for this experiment because tends to promote action-outcome learning and goal-directed decision making 38,39,42,86”
From A dual-pathway architecture for stress to disrupt agency and promote habit (2025) · cites this paper as [38]
An RI schedule of reinforcement tends to promote habit formation.
“We chose an RI reinforcement schedule for this experiment because it tends to promote habit formation 38,39,42,86”
From A dual-pathway architecture for stress to disrupt agency and promote habit (2025) · cites this paper as [38]
Cites
The Psychology of Animal Learning
Hill, W. F., & Mackintosh, N. J. (1976) · The American Journal of Psychology · doi:10.2307/1421765 · Cited by 2,397 · see citing works
The psychology of animal learning
Lea, S. E. G. (1975) · Animal Behaviour · doi:10.1016/0003-3472(75)90150-5 · Cited by 2,067 · see citing works
Instrumental Responding following Reinforcer Devaluation
Adams, C. D., & Dickinson, A. (1981) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748108400816 · Cited by 687 · see citing works
Self-citationVariations in the Sensitivity of Instrumental Responding to Reinforcer Devaluation
Adams, C. D. (1982) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748208400878 · Cited by 635 · see citing works
Self-citationOperant behavior as adaptation to constraint.
Staddon, J. E. R. (1979) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.108.1.48 · Cited by 335 · see citing works
Holland, P. C., & Straub, J. J. (1979) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.5.1.65 · Cited by 254 · see citing works
The temporal triangle: Response substitution in instrumental conditioning.
Rachlin, H. C., & Burkhard, B. (1978) · Psychological Review · doi:10.1037/0033-295x.85.1.22 · Cited by 155 · see citing works
Cue properties vs palatability of flavors in avoidance learning
García, J., Kovner, R., & Green, K. F. (1970) · Psychonomic Science · doi:10.3758/bf03329085 · Cited by 139 · see citing works
Acquisition of sodium by rats: Interaction of innate mechanisms and latent learning.
Krieckhaus, E. E., & Wolf, G. (1968) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0025547 · Cited by 138 · see citing works
"What is Learned?"—An empirical enigma.
Rozeboom, W. W. (1958) · Psychological Review · doi:10.1037/h0045256 · Cited by 106 · see citing works
YOKED VARIABLE‐RATIO AND VARIABLE‐INTERVAL RESPONDING IN PIGEONS1
Catania, A. C., Matthews, T. J., Silverman, P. J., & Yohalem, R. (1977) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1977.28-155 · Cited by 65 · see citing works
Some conditions for the dissociation of consummatory and instrumental behavior in rats
Holman, E. W. (1975) · Learning and Motivation · doi:10.1016/0023-9690(75)90015-6 · Cited by 58 · see citing works
Information Processing in Animals: Memory Mechanisms
Devine, J. V., Spear, N. E., & Miller, R. R. (1983) · The American Journal of Psychology · doi:10.2307/1422827 · Cited by 54 · see citing works
Post-Conditioning Devaluation of an Instrumental Reinforcer has no Effect on Extinction Performance
Adams, C. D. (1980) · Quarterly Journal of Experimental Psychology · doi:10.1080/14640748008401838 · Cited by 47 · see citing works
Self-citationTaste-mediated conditioned aversion to an exteroceptive stimulus following LiCl poisoning.
Morrison, G. R., & Collyer, R. (1974) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0035947 · Cited by 46 · see citing works
Chen, J.-S., & Amsel, A. (1980) · Science · doi:10.1126/science.7403850 · Cited by 27 · see citing works
THE STRUCTURE OF RESPONSE RATE1
Williams, D. R. (1968) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1968.11-251 · Cited by 22 · see citing works
Wilson, C. L., Sherman, J. E., & Holman, E. W. (1981) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.7.2.165 · Cited by 20 · see citing works
Iron appetite and latent learning in rats
Woods, S. C., Vasselli, J. R., & Milam, K. M. (1977) · Physiology & Behavior · doi:10.1016/0031-9384(77)90036-1 · Cited by 11 · see citing works
Taste aversion conditioning and t-maze behaviour
Jones, L. (1980) · Australian Journal of Psychology · doi:10.1080/00049538008254676 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785 · see citing works
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402 · see citing works
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Cardinal, R. N., Parkinson, J. A., Hall, J., & Everitt, B. J. (2002) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(02)00007-6 · Cited by 2,129 · see citing works
Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates
Dalley, J. W., Cardinal, R. N., & Robbins, T. W. (2004) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2004.09.006 · Cited by 1,329 · see citing works
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217 · see citing works
Self-citationYin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226 · see citing works
The role of the dorsomedial striatum in instrumental conditioning
Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04218.x · Cited by 1,099 · see citing works
Goal-directed and habitual control in the basal ganglia: implications for Parkinson's disease
Redgrave, P., Rodriguez, M., Smith, Y., Rodriguez-Oroz, M. C., Lehericy, S., Bergman, H., Agid, Y., DeLong, M. R., & Obeso, J. A. (2010) · Nature reviews. Neuroscience · doi:10.1038/nrn2915 · Cited by 1,134 · see citing works
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977 · see citing works
A specific role for posterior dorsolateral striatum in human habit learning
Tricomi, E., Balleine, B. W., & O’Doherty, J. P. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06796.x · Cited by 769 · see citing works
Moral outrage in the digital age
Crockett, M. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0213-3 · Cited by 618 · see citing works
Belin, D., Jonkman, S., Dickinson, A., Robbins, T. W., & Everitt, B. J. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.09.027 · Cited by 520 · see citing works
Self-citationLearning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475 · see citing works
Habitual alcohol seeking: time course and the contribution of subregions of the dorsal striatum
Corbit, L. H., Nie, H., & Janak, P. H. (2012) · Biological Psychiatry · doi:10.1016/j.biopsych.2012.02.024 · Cited by 486 · see citing works
Amphetamine Exposure Enhances Habit Formation
Nelson, A., & Killcross, S. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4305-05.2006 · Cited by 445 · see citing works
Associative theories of goal-directed behaviour: a case for animal–human translational models
Wit, S., & Dickinson, A. (2009) · Psychological Research Psychologische Forschung · doi:10.1007/s00426-009-0230-6 · Cited by 424 · see citing works
Self-citationRelations between Pavlovian-instrumental transfer and reinforcer devaluation
Holland, P. C. (2004) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/0097-7403.30.2.104 · Cited by 388 · see citing works
The integrative function of the basal ganglia in instrumental conditioning
Balleine, B. W., Liljeholm, M., & Ostlund, S. B. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.10.034 · Cited by 356 · see citing works
Goal-directed learning and obsessive-compulsive disorder
Gillan, C. M., & Robbins, T. W. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0475 · Cited by 348 · see citing works
Coutureau, E., & Killcross, S. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.025 · Cited by 342 · see citing works
Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits
BALLEINE, B. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.08.061 · Cited by 326 · see citing works
Shift from goal-directed to habitual cocaine seeking after prolonged experience in rats
Zapata, A., Minney, V. L., & Shippenberg, T. S. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4072-10.2010 · Cited by 296 · see citing works
Habits, action sequences and reinforcement learning
Dezfouli, A., & Balleine, B. W. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08050.x · Cited by 297 · see citing works
Model-based learning protects against forming habits
Gillan, C. M., Otto, A. R., Phelps, E. A., & Daw, N. D. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0347-6 · Cited by 283 · see citing works
The role of habit in compulsivity
Gillan, C. M., Robbins, T. W., Sahakian, B. J., van den Heuvel, O. A., & van Wingen, G. (2016) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2015.12.033 · Cited by 292 · see citing works
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280 · see citing works
Calculating Consequences: Brain Systems That Encode the Causal Effects of Actions
Tanaka, S. C., Balleine, B. W., & O'Doherty, J. P. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1808-08.2008 · Cited by 268 · see citing works
Alcohol seeking by rats: action or habit?
Dickinson, A., Wood, N., & Smith, J. W. (2002) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/0272499024400016 · Cited by 268 · see citing works
Self-citationHogarth, L., & Chase, H. W. (2011) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/a0022913 · Cited by 225 · see citing works
The role of cognition in classical and operant conditioning
Kirsch, I., Lynn, S. J., Vigorito, M., & Miller, R. R. (2004) · Journal of Clinical Psychology · doi:10.1002/jclp.10251 · Cited by 192 · see citing works
Endocannabinoid signaling is critical for habit formation
Hilário, M. R. F. (2007) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.006.2007 · Cited by 164 · see citing works
WICKENS, J. R., BUDD, C. S., HYLAND, B. I., & ARBUTHNOTT, G. W. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.016 · Cited by 165 · see citing works
On habits and addiction: an associative analysis of compulsive drug seeking
Ostlund, S. B., & Balleine, B. W. (2008) · Drug Discovery Today: Disease Models · doi:10.1016/j.ddmod.2009.07.004 · Cited by 147 · see citing works
A fundamental role for context in instrumental learning and extinction
Bouton, M. E., & Todd, T. P. (2014) · Behavioural Processes · doi:10.1016/j.beproc.2014.02.012 · Cited by 152 · see citing works
Memory systems and the addicted brain
Goodman, J., & Packard, M. G. (2016) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2016.00024 · Cited by 140 · see citing works
Impaired goal-directed behavioural control in human impulsivity
Hogarth, L., Chase, H. W., & Baess, K. (2012) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2010.518242 · Cited by 129 · see citing works
Dickinson, A. (1994) · Animal Learning and Cognition · doi:10.1016/b978-0-08-057169-0.50009-7 · Cited by 200 · see citing works
Self-citationInstrumental uncertainty as a determinant of behavior under interval schedules of reinforcement
DeRusso, A. L. (2010) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2010.00017 · Cited by 111 · see citing works
Learning to lose control: A process-based account of behavioral addiction
Perales, J. C., King, D. L., Navas, J. F., Schimmenti, A., Sescousse, G., Starcevic, V., van Holst, R. J., & Billieux, J. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.12.025 · Cited by 101 · see citing works
Context, attention, and the switch between habit and goal-direction in behavior
Bouton, M. E. (2021) · Learning & Behavior · doi:10.3758/s13420-021-00488-z · Cited by 100 · see citing works
Premotor cortex is critical for goal-directed actions
Gremel, C. M., & Costa, R. M. (2013) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2013.00110 · Cited by 75 · see citing works
Pool, E. R., Gera, R., Fransen, A., Perez, O. D., Cremer, A., Aleksic, M., Tanwisuth, S., Quail, S., Ceceli, A. O., Manfredi, D. A., et al. (2022) · Learning & Memory · doi:10.1101/lm.053413.121 · Cited by 74 · see citing works
Unexpected food outcomes can return a habit to goal-directed action
Bouton, M. E., Broomer, M. C., Rey, C. N., & Thrailkill, E. A. (2020) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2020.107163 · Cited by 65 · see citing works
Effects of the chronic restraint stress induced depression on reward-related learning in rats.
Xu, P., Wang, K., Lu, C., Dong, L., Chen, Y., Wang, Q., Shi, Z., Yang, Y., Chen, S., & Liu, X. (2017) · Behavioural Brain Research · doi:10.1016/j.bbr.2016.12.045 · Cited by 61 · see citing works
Some factors that restore goal-direction to a habitual behavior
Trask, S., Shipman, M. L., Green, J. T., & Bouton, M. E. (2020) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2020.107161 · Cited by 63 · see citing works
Shipman, M. L., Trask, S., Bouton, M. E., & Green, J. T. (2018) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2018.07.010 · Cited by 56 · see citing works
Aberrant habit formation in the Sapap3-knockout mouse model of obsessive-compulsive disorder
Hadjas, L. C., Lüscher, C., & Simmler, L. D. (2019) · Scientific Reports · doi:10.1038/s41598-019-48637-9 · Cited by 56 · see citing works
Simmler, L. D., & Ozawa, T. (2019) · Neurochemistry International · doi:10.1016/j.neuint.2019.104464 · Cited by 40 · see citing works
A dual-pathway architecture for stress to disrupt agency and promote habit
Giovanniello, J. R., Paredes, N., Wiener, A., Ramírez-Armenta, K., Oragwam, C., Uwadia, H. O., Yu, A. L., Lim, K., Pimenta, J. S., Vilchez, G. E., et al. (2025) · Nature · doi:10.1038/s41586-024-08580-w · Cited by 30 · see citing works
The Effect of Ratio and Interval Training on Pavlovian-Instrumental Transfer in Mice
Wiltgen, B. J., Sinclair, C., Lane, C., Barrows, F., Molina, M., & Chabanon-Hicks, C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0048227 · Cited by 28 · see citing works
Human habit neural circuitry may be perturbed in eating disorders
Wang, A. R., Kuijper, F. M., Barbosa, D. A. N., Hagan, K. E., Lee, E., Tong, E., Choi, E. Y., McNab, J. A., Bohon, C., & Halpern, C. H. (2023) · Science Translational Medicine · doi:10.1126/scitranslmed.abo4919 · Cited by 27 · see citing works
Revealing abrupt transitions from goal-directed to habitual behavior
Moore, S., Wang, Z., Zhu, Z., Wang, J., Sun, R., Lee, Y. A., Charles, A., & Kuchibhotla, K. V. (2026) · Nature Communications · doi:10.1038/s41467-026-71048-0 · Cited by 3 · see citing works
- Authors
- Dickinson, A., Nicholas, D. J., & Adams, C. D.
- Affiliations
- University of Cambridge (GB)
- Venue
- The Quarterly Journal of Experimental Psychology Section B
- Publisher
- SAGE Publications
- Published
- 1983-02
- Volume
- 35
- Issue
- 1b
- Pages
- 35-51
- ISSN
- 0272-4995 (ISSN-L)
- 1464-1321
- OpenAlex
- W2274685343
- Cited by
- 338
- References
- 20
- Citations by year
- 2026: 7 · 2025: 9 · 2024: 12 · 2023: 25 · 2022: 21 · 2021: 19 · 2020: 23 · 2019: 23 · 2018: 14 · 2017: 20 · 2016: 14 · 2015: 16 · 2014: 14 · 2013: 7 · 2012: 12
- FWCI
- 5.34
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Quarterly Journal of Experimental Psychology Section B (publishedVersion)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- reinforcer devaluation, operant conditioning, aversive conditioning, sucrose, extinction, interval schedules, lever pressing, rats
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
