The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Addiction as a Computational Process Gone Awry
Abstract
Addictive drugs have been hypothesized to access the same neurophysiological mechanisms as natural learning systems. These natural learning systems can be modeled through temporal-difference reinforcement learning (TDRL), which requires a reward-error signal that has been hypothesized to be carried by dopamine. TDRL learns to predict reward by driving that reward-error signal to zero. By adding a noncompensable drug-induced dopamine increase to a TDRL model, a computational model of addiction is constructed that over-selects actions leading to drug receipt. The model provides an explanation for important aspects of the addiction literature and provides a theoretic view-point with which to address other aspects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
A Theory of Rational Addiction
Becker, G. S., & Murphy, K. M. (1988) · Journal of Political Economy · doi:10.1086/261558 · Cited by 3,823
Drug Addiction, Dysregulation of Reward, and Allostasis
Koob, G. (2001) · Neuropsychopharmacology · doi:10.1016/s0893-133x(00)00195-0 · Cited by 2,953
Cocaine Receptors on Dopamine Transporters Are Related to Self-Administration of Cocaine
Ritz, M. C., Lamb, R. J., Goldberg, S. R., & Kuhar, M. J. (1987) · Science · doi:10.1126/science.2820058 · Cited by 2,180
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Context, ambiguity, and unlearning: sources of relapse after behavioral extinction
Bouton, M. E. (2002) · Biological Psychiatry · doi:10.1016/s0006-3223(02)01546-9 · Cited by 1,662
Robinson, T. E., & Berridge, K. C. (2003) · Annual Review of Psychology · doi:10.1146/annurev.psych.54.101601.145237 · Cited by 1,509
Cited by
NEURAL MECHANISMS OF ADDICTION: The Role of Reward-Related Learning and Memory
Hyman, S. E., Malenka, R. C., & Nestler, E. J. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113009 · Cited by 2,647
The debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
Habits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697
Paulus, M. P., & Stein, M. B. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.03.042 · Cited by 1,325
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
Addiction: A Disease of Learning and Memory
Hyman, S. E. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.8.1414 · Cited by 980
The Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928
New insights into symptoms and neurocircuit function of anorexia nervosa
Kaye, W. H., Fudge, J. L., & Paulus, M. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2682 · Cited by 843
From reinforcement learning models to psychiatric and neurological disorders
Maia, T. V., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2723 · Cited by 722
Belin, D., & Everitt, B. J. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.019 · Cited by 697
“During the development of drug addiction, drug seeking becomes progressively established as a maladaptive stimulusresponse (S-R) habit (Everitt et al., 2001; Everitt and Robbins, 2005; Volkow et al., 2006; Redish, 2004 ).”
Dopamine reward prediction error coding
Schultz, W. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/wschultz · Cited by 597
Lüscher, C., & Huber, K. M. (2010) · Neuron · doi:10.1016/j.neuron.2010.01.016 · Cited by 627
Intense Sweetness Surpasses Cocaine Reward
Lenoir, M., Serre, F., Cantin, L., & Ahmed, S. H. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000698 · Cited by 608
Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548
Bickel, W. K., Miller, M. L., Yi, R., Kowal, B. P., Lindquist, D. M., & Pitcock, J. A. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2006.09.016 · Cited by 571
A unified framework for addiction: Vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0800472x · Cited by 501
Self-citationReinforcement learning and episodic memory in humans and animals: an integrative framework
Gershman, S. J., & Daw, N. D. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033625 · Cited by 446
Resting state functional connectivity in addiction: Lessons learned and a road ahead
Sutherland, M. T., McHugh, M. J., Pariyadath, V., & Stein, E. A. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.117 · Cited by 494
Neurobiology of Decision Making: A Selective Review from a Neurocognitive and Clinical Perspective
Ernst, M., & Paulus, M. P. (2005) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.06.004 · Cited by 469
The computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 465
Luijten, M., Schellekens, A. F., Kühn, S., Machielse, M. W. J., & Sescousse, G. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.3084 · Cited by 402
Toward a neurobiology of delusions
Corlett, P. R., Taylor, J. R., Wang, X.-J., Fletcher, P. C., & Krystal, J. H. (2010) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2010.06.007 · Cited by 394
Relative reinforcing efficacy of alcohol among college student drinkers.
Murphy, J. G., & MacKillop, J. (2006) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.14.2.219 · Cited by 399
Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry
Keiflin, R., & Janak, P. H. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.037 · Cited by 382
Dopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381
“Dopamine has been implicated in a bewildering variety of processes and pathologies in the human brain; ranging from cortical excitability to attentional deficits [5], [6]; from motor control to akinesia and set switching deficits in Parkinson’s disease [7], [8], [9]; from working memory to schizophrenia [10], [11]; fr…”
BackgroundCraving's place in addiction theory: Contributions of the major models
Skinner, M. D., & Aubin, H.-J. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.11.024 · Cited by 372
A somatic marker theory of addiction
Verdejo-García, A., & Bechara, A. (2009) · Neuropharmacology · doi:10.1016/j.neuropharm.2008.07.035 · Cited by 363
Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346
“Addiction and relapse Addictive drugs have been hypothesized to drive maladaptive decision making through pharmacological interactions with neurophysiological mechanisms evolved for normal learning systems (Lowinson et al., 1997; Everitt et al., 2001; Kelley, 2004; Redish, 2004; Hyman, 2005).”
BackgroundSelf-citationInstant transformation of learned repulsion into motivational “wanting”
Robinson, M. J. F., & Berridge, K. C. (2013) · Current Biology · doi:10.1016/j.cub.2013.01.016 · Cited by 320
Wang, X.-J., & Krystal, J. H. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.018 · Cited by 312
A Neural Computational Model of Incentive Salience
Zhang, J., Berridge, K. C., Tindell, A. J., Smith, K. S., & Aldridge, J. W. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000437 · Cited by 299
Decision-Making Dysfunctions in Psychiatry—Altered Homeostatic Processing?
Paulus, M. P. (2007) · Science · doi:10.1126/science.1142997 · Cited by 297
NeuroEconomics: An overview from an economic perspective
Kenning, P., & Plassmann, H. (2005) · Brain Research Bulletin · doi:10.1016/j.brainresbull.2005.07.006 · Cited by 287
Model-based choices involve prospective neural activity
Doll, B. B., Duncan, K. D., Simon, D. A., Shohamy, D., & Daw, N. D. (2015) · Nature Neuroscience · doi:10.1038/nn.3981 · Cited by 283
Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors
Voon, V., Pessiglione, M., Brezing, C., Gallea, C., Fernandez, H. H., Dolan, R. J., & Hallett, M. (2010) · Neuron · doi:10.1016/j.neuron.2009.12.027 · Cited by 280
Tindell, A. J., Berridge, K. C., Zhang, J., Peciña, S., & Aldridge, J. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04411.x · Cited by 268
Anticipatory reward processing in addicted populations: a focus on the monetary incentive delay task
Balodis, I. M., & Potenza, M. N. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.08.020 · Cited by 267
“Theories, including reward deficiency syndrome (2), allostatic hypothesis (3), incentive salience (4), and temporal-difference reinforcement learning (5), emphasize hyporesponsiveness or hyperresponsiveness, particularly in the ventral striatum (VS), across reward processing phases and addiction stages.”
Neural signature of fictive learning signals in a sequential investment task
Lohrenz, T., McCabe, K., Camerer, C. F., & Montague, P. R. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0608842104 · Cited by 265
Dopamine, Time, and Impulsivity in Humans
Pine, A., Shiner, T., Seymour, B., & Dolan, R. J. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.6028-09.2010 · Cited by 300
“In principle, some of the aforementioned deficits can be related to dopaminergic effects by way of dopamine’s established role in reward learning (Redish, 2004; Frank et al., 2007; Dagher and Robbins, 2009).”
BackgroundCurrent Emotion Research in Health Behavior Science
Williams, D. M., & Evans, D. R. (2014) · Emotion Review · doi:10.1177/1754073914523052 · Cited by 263
Excessive cocaine use results from decreased phasic dopamine signaling in the striatum
Willuhn, I., Burgeno, L. M., Groblewski, P. A., & Phillips, P. E. M. (2014) · Nature Neuroscience · doi:10.1038/nn.3694 · Cited by 261
Ahmed, S. H., & Koob, G. F. (2005) · Psychopharmacology · doi:10.1007/s00213-005-2180-z · Cited by 254
Aberrant learning and memory in addiction
Torregrossa, M. M., Corlett, P. R., & Taylor, J. R. (2011) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2011.02.014 · Cited by 223
Stimulus-dependent dopamine release in attention-deficit/hyperactivity disorder.
Sikström, S., & Söderlund, G. (2007) · Psychological Review · doi:10.1037/0033-295x.114.4.1047 · Cited by 205
Dopamine signaling in the nucleus accumbens of animals self-administering drugs of abuse
Willuhn, I., Wanat, M. J., Clark, J. J., & Phillips, P. E. M. (2010) · Current Topics in Behavioral Neurosciences · doi:10.1007/7854_2009_27 · Cited by 193
Pathological circuit function underlying addiction and anxiety disorders
Lüthi, A., & Lüscher, C. (2014) · Nature Neuroscience · doi:10.1038/nn.3849 · Cited by 192
The research domain criteria framework: The case for anterior cingulate cortex
Holroyd, C. B., & Umemoto, A. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.021 · Cited by 187
Phasic dopamine release in appetitive behaviors and drug addiction
Wanat, M., Willuhn, I., Clark, J., & Phillips, P. (2009) · Current Drug Abuse Reviewse · doi:10.2174/1874473710902020195 · Cited by 177
Schultz, W. (2017) · Current Biology · doi:10.1016/j.cub.2017.02.064 · Cited by 177
Stuber, G. D., Wightman, R. M., & Carelli, R. M. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.036 · Cited by 168
Imaging brain response to reward in addictive disorders
Hommer, D. W., Bjork, J. M., & Gilman, J. M. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2010.05898.x · Cited by 163
Striatal sensitivity to reward deliveries and omissions in substance dependent patients
Bjork, J. M., Smith, A. R., & Hommer, D. W. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.06.035 · Cited by 154
Stimulus representation and the timing of reward-prediction errors in models of the dopamine system
Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2008) · Neural Computation · doi:10.1162/neco.2008.11-07-654 · Cited by 148
Phasic Nucleus Accumbens Dopamine Encodes Risk-Based Decision-Making Behavior
Sugam, J. A., Day, J. J., Wightman, R. M., & Carelli, R. M. (2012) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.09.029 · Cited by 142
Motivation and Self-Regulation in Addiction: A Call for Convergence
Köpetz, C. E., Lejuez, C. W., Wiers, R. W., & Kruglanski, A. W. (2013) · Perspectives on Psychological Science · doi:10.1177/1745691612457575 · Cited by 138
On the role of theory and modeling in neuroscience
Levenstein, D., Alvarez, V. A., Amarasingham, A., Azab, H., Chen, Z. S., Gerkin, R. C., Hasenstaub, A., Iyer, R., Jolivet, R. B., Marzen, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2003.13825 · Cited by 93
Self-citationIllicit dopamine transients: reconciling actions of abused drugs
Covey, D. P., Roitman, M. F., & Garris, P. A. (2014) · Trends in Neurosciences · doi:10.1016/j.tins.2014.02.002 · Cited by 90
Clark, L., & Zack, M. (2023) · Addictive Behaviors · doi:10.1016/j.addbeh.2023.107626 · Cited by 59
“However, when the relation between the CS and US is inconsistent, it is hypothesized that the CS evokes an initial phasic DA spike (RPE) and the US delivery continues to elicit a second RPE (Redish, 2004; Redish et al., 2007; Zack et al., 2020).”
BackgroundResultInfluentialCocaine addiction as a homeostatic reinforcement learning disorder.
Keramati, M., Durand, A., Girardeau, P., Gutkin, B., & Ahmed, S. H. (2017) · Psychological Review · doi:10.1037/rev0000046 · Cited by 44
Nicotinic alteration of decision-making
Naudé, J., Dongelmans, M., & Faure, P. (2015) · Neuropharmacology · doi:10.1016/j.neuropharm.2014.11.021 · Cited by 39
“acute injection, and during craving) is a clear challenge, but would provide valuable information on modifications of DA systems during nicotine exposure (Redish, 2004).”
InfluentialAhmed, S. H. (2018) · Pharmacology Biochemistry and Behavior · doi:10.1016/j.pbb.2017.07.005 · Cited by 39
Lee, A. M., Tai, L.-H., Zador, A., & Wilbrecht, L. (2015) · Neuroscience · doi:10.1016/j.neuroscience.2014.12.042 · Cited by 37
Effects of emotions on sexual behavior in men with and without hypersexuality
Miner, M. H., Dickenson, J., & Coleman, E. (2019) · Sexual Addiction & Compulsivity · doi:10.1080/10720162.2018.1564408 · Cited by 37
Abnormal reward valuation and event-related connectivity in unmedicated major depressive disorder
Rupprechter, S., Stankevicius, A., Huys, Q. J. M., Series, P., & Steele, J. D. (2021) · Psychological Medicine · doi:10.1017/s0033291719003799 · Cited by 27
“, 2011) and addiction (Redish, 2004; Redish, Jensen, & Johnson, 2008), implying different psychiatric disorders may reflect different disorders of neural valuation.”
Ribba, B. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.1094281 · Cited by 24
N. Kearns, D., A. Gomez-Serrano, M., & J. Tunstall, B. (2011) · Current Drug Abuse Reviewse · doi:10.2174/1874473711104040261 · Cited by 22
Biback, C., & Zack, M. (2015) · Current Addiction Reports · doi:10.1007/s40429-015-0064-9 · Cited by 20
Implications of research on comorbidity for the nature and management of substance misuse
Mueser, K. T., Kavanagh, D. J., & Brunette, M. F. (2007) · Translation of Addictions Science Into Practice · doi:10.1016/b978-008044927-2/50063-8 · Cited by 14
Addressing altered anticipation as a transdiagnostic target through computational psychiatry
Sepúlveda, P., Aitsahalia, I., Kumar, K., Atkin, T., & Iigaya, K. (2025) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2025.02.014 · Cited by 3
Metadata
- Authors
- Redish, A. D.
- Venue
- Science
- Published
- 2004-12-10
- Volume
- 306
- Issue
- 5703
- Pages
- 1944-1947
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2119631764
- Cited by
- 540
- References
- 36
- FWCI
- 8.31
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.