The DOI matches its Crossref record; 137 of the 137 references it cites were found in a registry or an index.
The Role of Learning in the Operation of Motivational Systems
Abstract
Acquired behavior is motivated by two forms of incentive learning. Pavlovian incentive learning reflects the acquisition of motivational properties by conditioned stimuli (CSs) through their association with appetitive and aversive reinforcers. Although the influence of appetitive CSs is modulated by primary motivational states, they exert a general motivation influence on appetitive behavior. By contrast, aversive CSs inhibit appetitive behavior. The second process, instrumental incentive learning, determines the incentive value assigned to outcomes of goal‐directed, instrumental action. This incentive value, and its control by primary motivational states, has to be learned through experience of the hedonic reactions elicited by the outcome. The two incentive learning processes function in parallel to motivate instrumental behavior.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 9,461
The neural basis of drug craving: An incentive-sensitization theory of addiction
Robinson, T. (1993) · Brain Research Reviews · doi:10.1016/0165-0173(93)90013-p · Cited by 7,346
The Psychology of Learning and Motivation
(2021) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(21)x0002-3 · Cited by 5,027
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
Skinner, B. F., & Hull, C. L. (1944) · The American Journal of Psychology · doi:10.2307/1416955 · Cited by 4,051
Integrative activity of the brain
Konorski, J. (1967) · University of Chicago Press eBooks · Cited by 2,222
Food reward: brain substrates of wanting and liking
Berridge, K. C. (1996) · Neuroscience & Biobehavioral Reviews · doi:10.1016/0149-7634(95)00033-b · Cited by 1,901
Context, time, and memory retrieval in the interference paradigms of Pavlovian learning
Bouton, M. E. (1993) · Psychological Bulletin · doi:10.1037/0033-2909.114.1.80 · Cited by 1,896
Psychology of Learning and Motivation
Bower, G. H. (1975) · Cited by 1,843
Mowrer, O. H. (1960) · doi:10.1037/10802-000 · Cited by 1,670
Cited by
Reflective and Impulsive Determinants of Social Behavior
Strack, F., & Deutsch, R. (2004) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0803_1 · Cited by 3,984
“In particular, we propose that whenever a state of deprivation is successfully abolished, behaviors and situational circumstances that led to satisfying the need become strongly linked to the experience of this deprivation (cf. Dickinson & Balleine, 2002).”
BackgroundThe debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Balleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775
Self-citationBerridge, K. C., & Robinson, T. E. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00233-9 · Cited by 1,869
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
A new look at habits and the habit-goal interface.
Wood, W., & Neal, D. T. (2007) · Psychological Review · doi:10.1037/0033-295x.114.4.843 · Cited by 1,470
“This analysis builds on instrumental learning theories in which habits develop as organisms learn context–response associations in order to obtain rewarding events (see Dickinson & Balleine, 2002 ).”
BackgroundMethodInfluentialMotivation concepts in behavioral neuroscience
Berridge, K. C. (2004) · Physiology & Behavior · doi:10.1016/j.physbeh.2004.02.004 · Cited by 1,302
Affective neuroscience of pleasure: reward in humans and animals
Berridge, K. C., & Kringelbach, M. L. (2008) · Psychopharmacology · doi:10.1007/s00213-008-1099-6 · Cited by 1,254
“Liking” and “wanting” food rewards: Brain substrates and roles in eating disorders
Berridge, K. C. (2009) · Physiology & Behavior · doi:10.1016/j.physbeh.2009.02.044 · Cited by 1,147
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133
The role of the dorsomedial striatum in instrumental conditioning
Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04218.x · Cited by 1,099
Self-citationDolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
The cognitive functions of the caudate nucleus
Grahn, J. A., Parkinson, J. A., & Owen, A. M. (2008) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2008.09.004 · Cited by 940
LeDoux, J. E. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1400335111 · Cited by 891
The tempted brain eats: Pleasure and desire circuits in obesity and eating disorders
Berridge, K. C., Ho, C.-Y., Richard, J. M., & DiFeliceantonio, A. G. (2010) · Brain Research · doi:10.1016/j.brainres.2010.04.003 · Cited by 873
A specific role for posterior dorsolateral striatum in human habit learning
Tricomi, E., Balleine, B. W., & O’Doherty, J. P. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06796.x · Cited by 769
“R) habit has been formed, a cue will elicit its associated response regardless of outcome value (Dickinson & Balleine, 2002).”
BackgroundMethodSelf-citationReinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 717
From prediction error to incentive salience: mesolimbic computation of reward motivation.
Berridge, K. C. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.07990.x · Cited by 700
Parallel incentive processing: an integrated view of amygdala function
Balleine, B. W., & Killcross, S. (2006) · Trends in Neurosciences · doi:10.1016/j.tins.2006.03.002 · Cited by 622
Self-citationHyperdopaminergic mutant mice have higher "wanting" but not "liking" for sweet rewards
Peciña, S., Cagniard, B., Berridge, K. C., Aldridge, J. W., & Zhuang, X. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-28-09395.2003 · Cited by 551
Goal Contagion: Perceiving Is for Pursuing.
Aarts, H., Gollwitzer, P. M., & Hassin, R. R. (2004) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.87.1.23 · Cited by 512
Reinforcement learning and episodic memory in humans and animals: an integrative framework
Gershman, S. J., & Daw, N. D. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033625 · Cited by 446
Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis
Huys, Q. J., Pizzagalli, D. A., Bogdan, R., & Dayan, P. (2013) · Biology of Mood & Anxiety Disorders · doi:10.1186/2045-5380-3-12 · Cited by 480
“Alternative models One advantage of the simplicity of the task is that it is likely insensitive to several aspects of reinforcement learning that are under current investigation, such as goal-directed versus habitual decision-making [117,118].”
BackgroundHuys, Q. J. M., Eshel, N., O'Nions, E., Sheridan, L., Dayan, P., & Roiser, J. P. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002410 · Cited by 439
The dark side of emotion: the addiction perspective
Koob, G. F. (2015) · European Journal of Pharmacology · doi:10.1016/j.ejphar.2014.11.044 · Cited by 433
“Modern incentive-motivation theory, which has guided the neurobiology of motivation, has many contributions that range from two-process learning theory to instrumental incentive learning (Rescorla and Solomon, 1967; Dickinson and Balleine, 2002).”
BackgroundYin, H. H., Ostlund, S. B., & Balleine, B. W. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06422.x · Cited by 416
Self-citationRelations between Pavlovian-instrumental transfer and reinforcer devaluation
Holland, P. C. (2004) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/0097-7403.30.2.104 · Cited by 388
Serotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381
Palmiter, R. D. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1417.003 · Cited by 375
Otto (2013) · doi:10.1177/0956797612463080 · Cited by 352
“…in typical formulations, it is often unclear which system produced a given behavior, and the contributions of each system can often be conclusively distinguished only by procedures that are both laborious and theory dependent (Dickinson & Balleine, 2004; Gläscher, Daw, Dayan, & O’Doherty, 2010).”
BackgroundMethodInfluentialArana, F. S., Parkinson, J. A., Hinton, E., Holland, A. J., Owen, A. M., & Roberts, A. C. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-29-09632.2003 · Cited by 345
Three Pillars for the Neural Control of Appetite
Sternson, S. M., & Eiselt, A.-K. (2017) · Annual Review of Physiology · doi:10.1146/annurev-physiol-021115-104948 · Cited by 343
Pavlovian to instrumental transfer: A neurobehavioural perspective
Holmes, N. M., Marchand, A. R., & Coutureau, E. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.03.007 · Cited by 334
Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits
BALLEINE, B. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.08.061 · Cited by 326
Self-citationPredictive representations can link model-based reinforcement learning to model-free mechanisms
Russek, E. M., Momennejad, I., Botvinick, M. M., Gershman, S. J., & Daw, N. D. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005768 · Cited by 326
Chase, H. W., Eickhoff, S. B., Laird, A. R., & Hogarth, L. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.05.025 · Cited by 316
Mapping value based planning and extensively trained choice in the human brain
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3068 · Cited by 310
The basolateral amygdala in reward learning and addiction
Wassum, K. M., & Izquierdo, A. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.08.017 · Cited by 306
Process Account of Curiosity and Interest: A Reward-Learning Perspective
Murayama, K., FitzGibbon, L., & Sakaki, M. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09499-9 · Cited by 306
Murayama, K. (2022) · Psychological Review · doi:10.1037/rev0000349 · Cited by 304
Rethinking avoidance: Toward a balanced approach to avoidance in treating anxiety disorders
Hofmann, S. G., & Hay, A. C. (2018) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.03.004 · Cited by 303
A Neural Computational Model of Incentive Salience
Zhang, J., Berridge, K. C., Tindell, A. J., Smith, K. S., & Aldridge, J. W. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000437 · Cited by 299
Human Pavlovian–Instrumental Transfer
Talmi, D., Seymour, B., Dayan, P., & Dolan, R. J. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4028-07.2008 · Cited by 301
The role of dopamine in the accumbens core in the expression of Pavlovian‐conditioned responses
Saunders, B. T., & Robinson, T. E. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08217.x · Cited by 299
Dopamine enhances model-based over model-free choice behavior
Wunderlich, K., Smittenaar, P., & Dolan, R. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.042 · Cited by 294
“By comparison, model-based choice involves forward planning, inwhich an agent searches a cognitive model of the environment to find the same optimal actions (Dickinson and Balleine, 2002).”
BackgroundBlaisdell, A. P., Sawa, K., Leising, K. J., & Waldmann, M. R. (2006) · Science · doi:10.1126/science.1121872 · Cited by 287
Evolving Concepts of Emotion and Motivation
Berridge, K. C. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01647 · Cited by 273
Dopamine restores reward prediction errors in old age
Chowdhury, R., Guitart-Masip, M., Lambert, C., Dayan, P., Huys, Q., Düzel, E., & Dolan, R. J. (2013) · Nature Neuroscience · doi:10.1038/nn.3364 · Cited by 273
Serotonin, inhibition, and negative mood
Dayan, P., & Huys, Q. J. M. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040004 · Cited by 263
“First, 5-HT is involved in the prediction of aversive events, possibly as a form of opponent [1–3] to dopamine [4–11].”
Alcohol seeking by rats: action or habit?
Dickinson, A., Wood, N., & Smith, J. W. (2002) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/0272499024400016 · Cited by 268
Self-citationTindell, A. J., Berridge, K. C., Zhang, J., Peciña, S., & Aldridge, J. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04411.x · Cited by 268
Evolving the future: Toward a science of intentional change
Wilson, D. S., Hayes, S. C., Biglan, A., & Embry, D. D. (2014) · Behavioral and Brain Sciences · doi:10.1017/s0140525x13001593 · Cited by 255
A normative perspective on motivation
NIV, Y., JOEL, D., & DAYAN, P. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.06.010 · Cited by 251
Social reward among juvenile mice
Panksepp, J. B., & Lahvis, G. P. (2007) · Genes, Brain and Behavior · doi:10.1111/j.1601-183x.2006.00295.x · Cited by 248
Homeostatic reinforcement learning for integrating reward collection and physiological stability
Keramati, M., & Gutkin, B. (2014) · eLife · doi:10.7554/elife.04811 · Cited by 218
Retrospective revaluation in sequential decision making: a tale of two systems
Gershman, S. J., Markman, A. B., & Otto, A. R. (2014) · Journal of Experimental Psychology General · doi:10.1037/a0030844 · Cited by 217
Striatal Dopamine and the Interface between Motivation and Cognition
Aarts, E., van Holstein, M., & Cools, R. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00163 · Cited by 211
Corticostriatal Control of Goal-Directed Action Is Impaired in Schizophrenia
Morris, R. W., Quail, S., Griffiths, K. R., Green, M. J., & Balleine, B. W. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.06.005 · Cited by 197
Self-citationTwenty-Five Lessons from Computational Neuromodulation
Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190
Basic emotions or ur-emotions?
Frijda, N. H., & Parrott, W. G. (2011) · Emotion Review · doi:10.1177/1754073911410742 · Cited by 180
Basolateral amygdala to orbitofrontal cortex projections enable cue-triggered reward expectations
Lichtenberg, N. T., Pennington, Z. T., Holley, S. M., Greenfield, V. Y., Cepeda, C., Levine, M. S., & Wassum, K. M. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.0486-17.2017 · Cited by 171
Endocannabinoid signaling is critical for habit formation
Hilário, M. R. F. (2007) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.006.2007 · Cited by 164
“…seems to be governed more by stimulus-response than action-outcome relations because responding in trained animals become less sensitive to devaluation, and they do generalize to an identical lever that never lead to delivery of the outcome (Dickinson, 1985; Dickinson and Balleine, 1995, 2002).”
On habits and addiction: an associative analysis of compulsive drug seeking
Ostlund, S. B., & Balleine, B. W. (2008) · Drug Discovery Today: Disease Models · doi:10.1016/j.ddmod.2009.07.004 · Cited by 147
Self-citationEitam, B., Kennedy, P. M., & Tory Higgins, E. (2013) · Experimental Brain Research · doi:10.1007/s00221-012-3370-7 · Cited by 141
Meta-analysis of human prediction error for incentives, perception, cognition, and action
Corlett, P. R., Mollick, J. A., & Kober, H. (2022) · Neuropsychopharmacology · doi:10.1038/s41386-021-01264-3 · Cited by 135
Dynamic Computation of Incentive Salience: “Wanting” What Was Never “Liked”
Tindell, A. J., Smith, K. S., Berridge, K. C., & Aldridge, J. W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2499-09.2009 · Cited by 124
“…salience hypothesis, this anticipatory motivational revaluation occurs because the Pavlovian CS has acquired incentive-relevant features of its UCS, allowing the CS to interact with physiological states as would its UCS (Bindra, 1978; Toates, 1986; Dickinson and Balleine, 2002; Berridge, 2004).”
BackgroundInfluentialDopamine, Prediction Error and Beyond
Diederen, K. M. J., & Fletcher, P. C. (2021) · The Neuroscientist · doi:10.1177/1073858420907591 · Cited by 106
Learning to lose control: A process-based account of behavioral addiction
Perales, J. C., King, D. L., Navas, J. F., Schimmenti, A., Sescousse, G., Starcevic, V., van Holst, R. J., & Billieux, J. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.12.025 · Cited by 101
Hogarth, L. (2012) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/a0028914 · Cited by 99
Psychological Correlates of Ghosting and Breadcrumbing Experiences: A Preliminary Study among Adults
Navarro, R., Larrañaga, E., Yubero, S., & Víllora, B. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17031116 · Cited by 79
Carruthers, P., & Williams, D. M. (2022) · Cognition · doi:10.1016/j.cognition.2022.105117 · Cited by 71
Alvares, G. A., Balleine, B. W., & Guastella, A. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0094778 · Cited by 63
Self-citationCooper, R. P., Ruh, N., & Mareschal, D. (2014) · Cognitive Science · doi:10.1111/cogs.12067 · Cited by 51
“Indeed, while action selection within the contention scheduling system is model-free, it may still be necessary to learn a model of the environment in order to predict feedback (see Dickinson & Balleine, 2002).”
Ågmo, A., & Laan, E. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104595 · Cited by 42
The Effect of Ratio and Interval Training on Pavlovian-Instrumental Transfer in Mice
Wiltgen, B. J., Sinclair, C., Lane, C., Barrows, F., Molina, M., & Chabanon-Hicks, C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0048227 · Cited by 28
Caution influences avoidance and approach behaviors differently
Zhou, J., Hormigo, S., Sajid, M. S., & Castro-Alamancos, M. A. (2022) · The Journal of Neuroscience · doi:10.1523/jneurosci.1892-21.2022 · Cited by 21
Excessive appetitive arousal in Prader–Willi syndrome
Hinton, E. C., Isles, A. R., Williams, N. M., & Parkinson, J. A. (2010) · Appetite · doi:10.1016/j.appet.2009.12.002 · Cited by 19
Metadata
- Authors
- Dickinson, A., & Balleine, B.
- Venue
- Stevens' Handbook of Experimental Psychology
- Publisher
- Wiley
- Published
- 2002-02-21
- OpenAlex
- W2131938106
- Cited by
- 384
- References
- 137
- Type
- other
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.